Statesman roadmap

Statesman is being developed as a small authoritative core surrounded by optional capabilities. A capability only belongs in the shared abstraction when every provider can make the same promise. Stronger guarantees are exposed through separate interfaces and provider negotiation rather than implied by a convenient method name.

0.1: executable declaration and local authority

The repository currently implements the first coherent vertical slice:

  • immutable, fingerprinted declarations and typed state keys
  • root, container, and partition boundaries
  • proactive whole-state and facet loaders
  • reactive set, update, invalidate, clear, and typed interactions
  • current, stale-aware, and forced-fresh reads
  • optimistic revisions, single-flight refresh, and local coherent capture
  • append-only history with revision, age, byte, and tombstone retention
  • in-memory, filesystem, Redis, EF Core, and tiered stores
  • typed and root observation streams plus operation subscriptions
  • ASP.NET Core inspection and signal endpoints
  • mutation and key-determinism analyzers
  • deterministic test harnesses, provider tests, end-to-end tests, CI, and packaging

This release is intentionally a preview. Its public concepts are deliberate, but package compatibility is not promised until the 1.0 criteria are met.

0.2: contract hardening

The next milestone focuses on proving behavior across providers rather than adding breadth.

  • shared provider conformance suite covering compare-and-swap, exact imports, retention, ordering, cancellation, and corruption behavior — the compare-and-swap, cancellation, import-rejection, partition-discovery, exact-position-import, provider push notification, and distributed capture contracts shipped as Phase 16 items in the 0.3 design spec: tests/Statesman.Conformance.Tests now carries one shared abstract suite per contract (LedgerWriteConformanceTests, CancellationConformanceTests, ImportRejectionConformanceTests — the corruption contract's import-refusal half — PartitionCatalogConformanceTests, LedgerReplicaConformanceTests, ChangeNotifierConformanceTests, DistributedCaptureConformanceTests), each run against all five built-in providers (in-memory, filesystem, Redis, Entity Framework Core, tiered) through one construction point, ConformanceProviders, alongside the change-feed and lease suites already sharing that fixture. Every suite carries a break-the-mechanism proof pairing a wrong double that fails the shared assertion with a correct one that passes, and every capability gap is an honest Assert.SkipUnless skip rather than a silent pass — filesystem and Entity Framework Core skip the notifier suite (no cross-process push primitive), in-memory and filesystem skip distributed capture (no cross-process guarantee to offer), and the tiered provider skips every import-only fact (it vetoes IStateLedgerReplica outright). Retention semantics are now in the shared suite too: RetentionConformanceTests pins fourteen facts across all five built-in providers, with its discrimination proof BrokenRetentionConformanceTests's four narrow wrong doubles and one correct half, covering the newest-N MaxRevisions rule, the inclusive MaxAge cutoff, newest-first MaxBytes admission with its non-contiguous survivor set, payload-only byte accounting, the sequential narrowing of MaxRevisions, MaxAge, and MaxBytes rather than independent intersection, the latest-revision tombstone exemption, prune idempotence, the unknown-address no-op, and feed, head, and partition-catalog parity after a prune on every limit; provider-specific corruption behaviour beyond import refusal (a torn file, a truncated stream) stays deliberately per-provider, because each provider's failure shape is a property of its own storage format. Statesman.Conformance.Tests reached total: 312 (228 succeeded, 84 honest skips without live Redis; 288 succeeded, 24 skips with it), failed: 0 on SQLite and live Redis alike.
  • load diagnostics with structured per-source timing and health summaries — shipped as a Phase 17 item in the 0.3 design spec: RuntimeDefinition.LoadAsync times the whole load and every source off the runtime's own TimeProvider, stamping three new reserved, bounded record-metadata keys (statesman.load.duration.ms, statesman.load.started, statesman.load.completed) on every completed load (complete, seeded, retained, partial and initial-fallback alike; a load that throws outright records none of the three), with the structured half — StateSourceLoadReport, StateLoadReport, LoadDiagnostics in Statesman.Abstractions — read through two new IStatesmanDiagnostics members, ReadLoadDiagnostics()/ClearLoadDiagnostics(), retained latest-per-address and bounded at StatesmanDiagnostics.MaxRetainedLoadReports (64). A new meter instrument, statesman.load.source.duration, carries the per-source breakdown instead: per-source timing is deliberately kept off the record because a per-source metadata key is unbounded in the number of declared sources, where the three record keys above are bounded regardless of source count. StatesmanHealthCheck gains five new Data keys derived from the load-diagnostics surface (statesman.load.reports, statesman.load.reports.incomplete, statesman.load.sources.faulted, statesman.load.slowest.source, statesman.load.slowest.duration.ms) and a third Degraded condition — a retained report whose completeness is partial or initial-fallback — joining the two Phase 16 conditions; it self-corrects because the next complete refresh of that address replaces the degraded report. A new shared conformance suite, RecordMetadataConformanceTests, pins that StateRecord.Metadata — including the three reserved timing keys — survives append, both reads, and an exact import on all five built-in providers.
  • OpenTelemetry semantic conventions, health checks, and rate-limited maintenance failure reporting — shipped as Phase 16 items in the 0.3 design spec: a per-store token bucket (StatesmanDiagnostics.MaintenanceFailureRate = 16 per StatesmanDiagnostics.MaintenanceFailureRateWindow = one minute) now gates maintenance-failure retention ahead of the existing 64-entry bound, reported through a new statesman.maintenance.failures.suppressed counter alongside the two Phase 15 counters, and reachable through a new public surface, IStatesmanDiagnostics (ReadMaintenanceFailures(), ClearMaintenanceFailures()), discovered via StatesmanDiagnosticsExtensions.TryGetDiagnostics rather than a new IStatesman member, so no baselined interface breaks. StatesmanHealthCheck (Statesman.Extensions.Hosting) reports degraded when maintenance failures are retained (readable through the diagnostics surface), or a store runs interval maintenance without a lease, and unhealthy while a root has not finished initializing; because IHealthChecksBuilder/AddCheck live only in the larger, non-abstractions health-checks package and not the .Abstractions package Statesman.Extensions.Hosting references, it ships without an AddStatesmanHealthCheck extension — register with services.AddHealthChecks().AddCheck<StatesmanHealthCheck>("statesman"). A new pinning test, StatesmanTelemetryConventionTests, asserts the exact name, kind, unit and tag-key set of every documented instrument on the Statesman meter, discovered through the meter's own static initializer rather than a test-created decoy, so a renamed or re-unit'd instrument fails a named test instead of drifting silently — this is the OpenTelemetry semantic-conventions check, documented alongside two accepted 0.x deviations on the new docs/reference/telemetry.md.
  • serializer envelopes with content type, serializer id, and migration provenance — shipped as a Phase 22 item in the 0.3 design spec: StateEnvelope, a public, non-positional record in Statesman.Abstractions (ContentType and SerializerId required, Fingerprint nullable, FormatVersion defaulting to StateEnvelope.CurrentFormatVersion = 1), and one nullable Envelope member added to each of StateRecord and StateCommit; IStateSerializer gains SerializerId and ContentType as default interface members, defaulting to the implementing type's full name and application/octet-stream, with JsonStateSerializer overriding both with the persisted contract statesman.json/v1 / application/json — a stable identity from this release on, never renamed. Every write point on StateHandle stamps the envelope by default, answering null for a commit with no payload, so no opt-in switch is needed the way RedisKeyLayout needed one: this change moves no existing data. Persisted per provider with no migration except one: the filesystem store and Statesman.Tooling's export/restore carry it as a nested object, with no bump to the export format version; each of the three Entity Framework Core ledger packages (Statesman.Persistence.EntityFrameworkCore.{Sqlite,SqlServer,PostgreSQL}) gains one nullable EnvelopeJson JSON column per ledger entity and one generated migration — 20260915204340_SerializerEnvelope (SQLite), 20260915204350_SerializerEnvelope (SQL Server), 20260915204358_SerializerEnvelope (PostgreSQL) — while the outbox context's own three packages are untouched, measured unnecessary; Redis carries it as a member of the existing record blob, declared before GlobalPosition, which is load-bearing rather than tidiness (GlobalPosition must stay the JSON document's last property for the append script's position-suffix trick); Statesman.Tooling carries it verbatim through an export and a restore. Migration provenance is the declaration fingerprint the outbox already asserted, plus the envelope's own FormatVersion beside the record's existing SchemaVersion — there is no fifth member, because the write path serializes a value the caller supplies, which the runtime cannot honestly attribute to a prior migration. Nothing validates an envelope on read, anywhere, including in Statesman.Tooling's restore: this phase carries the value, it does not check it.
  • filesystem recovery tooling — verify and repair shipped as a Phase 18 item in the 0.3 design spec: FileSystemStateLedgerStore.VerifyAsync reports, without writing anything, every instance of nine finding kinds (a torn or malformed change-log line, a dangling or position-mismatched change-log line, a missing head or history file, an unreadable record file, an orphaned temporary file, a misplaced stream directory); RepairAsync (dry-run by default, the opposite of CompactChangeLogAsync()) acts on five of them — writing a head or history file back from data already in the same stream directory, quarantining an unreadable file by rename, and dropping a dangling or position-mismatched change-log line through the existing CompactChangeLogAsync — and never touches the other four (a torn tail, a non-final malformed line, an orphaned temporary file, a misplaced stream directory), on the rule that repair never destroys data verify could not prove orphaned; change-log compaction shipped separately in 0.3. docs/operations/recovery.md is the new cross-provider corruption runbook this bullet closes "upgrade and corruption scenarios are documented" for
  • Redis cluster and script compatibility tests — cluster coverage shipped as a Phase 21 item in the 0.3 design spec: RedisStateLedgerStoreOptions.KeyLayout, an opt-in RedisKeyLayout enum defaulting to Legacy (byte-for-byte the keys every previous release wrote, so no existing deployment's data moves), with SingleSlot wrapping every key of one store in the hash tag {KeyPrefix:Name} so the six-key append script and multi-address distributed capture are legal on a cluster; a new redis-cluster CI job runs Statesman.Conformance.Tests, Statesman.Redis.Tests, Statesman.Tooling.Tests and Statesman.Outbox.Redis.Tests against a single-node redis:7-alpine cluster with SingleSlot selected, measured to take the shared conformance suite from 47 CROSSSLOT-caused failures under Legacy to zero, identical to its standalone result; Statesman.Outbox.Redis needed no option, since every one of its keys is already single-key and it passes 17 of 17 against the cluster unchanged. Multi-node MOVED/ASK redirection is a recorded parked item rather than an open gap: a single-node cluster exercises CROSSSLOT and Lua slot validation, the whole of the key-layout question, and redirection needs at least three masters and a materially larger CI job for coverage of a reply StackExchange.Redis already follows itself.
  • EF Core migrations for SQL Server, PostgreSQL, and SQLite shipped in 0.3, as six packages — Statesman.Persistence.EntityFrameworkCore.{Sqlite,SqlServer,PostgreSQL} and Statesman.Outbox.EntityFrameworkCore.{Sqlite,SqlServer,PostgreSQL}, three per context because the three engines share no type mapping — and retry-on-failure became a supported configuration there rather than guidance. Built as a Phase 14 item in the 0.3 design spec: MigrationsAssembly and MigrationsHistoryTable as the seam, a baseline history row — not a baseline migration — for consumers already on EnsureCreated, and an xUnit drift test — not dotnet ef migrations has-pending-model-changes in CI — as the exit criterion. Shipped migrations stay opt-in; the consumer-owns-the-migration arrangement both contexts document today remains the supported default.
  • analyzer code fixes and stronger collection-mutation data-flow analysis — shipped as a Phase 20 item in the 0.3 design spec: the bullet named three rules and one new package; what shipped is four rules and one new project inside the existing Statesman.Analyzers package. Statesman.CodeFixes, a new netstandard2.0 project referencing Microsoft.CodeAnalysis.CSharp.Workspaces (the one central pin RS1038 forces out of the analyzer assembly itself), ships its DLL inside the existing analyzer nupkg under analyzers/dotnet/cs rather than as a second package, so dotnet pack Statesman.slnx -c Release still produces exactly 22 packages. STM002 gains two code fixes with fix-all support — Make this property init-only and Make this field readonly. STM004 is a new rule reporting an in-place mutation of a collection reached through a [ManagedState]-owned member, on a receiver-chain walk with no data flow and no alias tracking; the alias gap (var items = state.Items; items.Add(1)) is a deliberate design ruling rather than an open gap, and is documented in the analyzer guide's limitations sentence. STM001's ownership resolution is widened to the same receiver-chain walk — a behaviour change to a shipped rule, recorded as a ### Changed entry — so the two rules no longer disagree about the same managed member. STM003 ships a "how to fix" paragraph and a help link rather than a code fix, because constant folding already silences every foldable argument shape and the one remaining mechanically fixable shape has zero occurrences in this repository. All four rules now carry a helpLinkUri to docs/guides/analyzers.md. Statesman.Analyzers.Tests rose from a measured baseline of total: 3 to total: 53, failed: 0.
  • public API compatibility baselines and package-validation gates — shipped as a Phase 15 item in the 0.3 design spec: PackageValidationBaselineVersion baselines the fifteen packages v0.3.0 published, at 0.3.0, set once in src/Directory.Build.props; the seven packages with no 0.3.0 release — Statesman.Outbox.EntityFrameworkCore, added after the tag, and the six Phase 14 migration packages — blank the property in their own project file. The two deliberate breaks this baseline records, IStateChangeFeed.ReadAsync (0.3's previous phase) and Statesman.Outbox.OutboxCursorFile (this phase), are recorded as seven committed CompatibilitySuppressions.xml files, and every entry is explained on the new docs/reference/api-compatibility.md rather than in the generated XML, because a hand-added XML comment does not survive regeneration. The gate runs on dotnet pack in both ci.yml's pack job and release.yml, with no workflow edit, because both already restore before packing.

With bullets 1 and 3 substantially shipped in Phase 16 and bullet 2 shipped in Phase 17, every ROADMAP 0.2 bullet has been addressed, though not every one is complete. Bullet 1 (shared provider conformance) still runs one suite across all five built-in providers for compare-and-swap, cancellation, import rejection (the corruption contract's import-refusal half), partition discovery, exact-position import, provider push notification, and distributed capture, now joined by a record-metadata round-trip suite — total: 312, 228 succeeded / 84 honest skips without live Redis, 288 / 24 with it, failed: 0 on SQLite and live Redis, and on SQL Server and PostgreSQL for the four Entity Framework Core-affected projects — retention semantics (MaxAge/MaxRevisions/MaxBytes pruning) are now in the shared suite too, pinned by RetentionConformanceTests and its discrimination proof BrokenRetentionConformanceTests, and provider-specific corruption beyond import refusal stays deliberately per-provider. Bullet 2 (load diagnostics with structured per-source timing and health summaries) now ships: three bounded record-metadata timing keys, a typed StateLoadReport/LoadDiagnostics surface bounded at 64 retained reports, a new statesman.load.source.duration instrument carrying the per-source breakdown the record deliberately does not, and five new StatesmanHealthCheck Data keys with a third Degraded condition for an incomplete latest load. Bullet 3 (OpenTelemetry semantic conventions, health checks, and rate-limited maintenance failure reporting) is fully shipped, unchanged since Phase 16: a per-store rate limiter ahead of the existing retention bound, a public IStatesmanDiagnostics surface, StatesmanHealthCheck, and a pinning test over the meter's documented instruments and tag keys. Bullet 4 (serializer envelopes) now ships, as a Phase 22 item in the 0.3 design spec: StateEnvelope (ContentType, SerializerId, nullable Fingerprint, FormatVersion) is purely additive on both StateRecord and StateCommit — the whole solution compiles unchanged and an un-enveloped Redis member is byte-identical to what earlier releases wrote — stamped by every write point, persisted by every built-in provider with a single generated Entity Framework Core migration per engine and no other data movement, and read back unchecked: nothing validates an envelope's contents against the serializer in use this phase. Bullet 5 (filesystem recovery tooling) now ships, as a Phase 18 item in the 0.3 design spec: FileSystemStateLedgerStore.VerifyAsync reports all nine finding kinds without writing anything; RepairAsync (dry-run by default) writes a head or history file back from data already in the same stream directory, quarantines an unreadable record file by rename, and drops a dangling or position-mismatched change-log line through the existing CompactChangeLogAsync, on the rule that repair never destroys data verify could not prove orphaned — and deliberately leaves a torn tail, a non-final malformed line, an orphaned temporary file, and a misplaced stream directory alone, each for its own documented reason; docs/operations/recovery.md is the new cross-provider restore-from-corruption runbook. Bullet 6 (Redis cluster coverage) now ships, as a Phase 21 item in the 0.3 design spec: an opt-in RedisKeyLayout.SingleSlot hash-tags one store's keys into a single Redis Cluster hash slot, defaulting to Legacy so no existing deployment moves, and a redis-cluster CI job proves it against a single-node redis:7-alpine cluster running the four Redis-affected suites; multi-node MOVED/ASK redirection is a recorded parked item, not an open gap. Bullet 8 (analyzer code fixes) now ships, as a Phase 20 item in the 0.3 design spec: STM002's two code fixes with fix-all, the new STM004 collection-mutation rule, STM001's widened receiver-chain ownership, and STM003's documentation and help link, shipped from one new project packaged inside the existing analyzer nupkg rather than as a second package. Alias tracking for STM004, recorded in Phase 20 as a design ruling rather than an open gap, itself shipped in Phase 21 for both STM001 and STM004, alongside a widened STM004 membership rule and mutator set and conversion arms on the shared ownership walk — none of which change this bullet's shipped status, already true since Phase 20. The 0.3 design spec's Phase 18, Phase 20 and Phase 21 sections list what is parked and why, under "Explicitly parked, with reasons". Of the exit criteria below: "CI builds every supported target framework" is fully met; "all built-in providers pass one conformance suite" is met, on addendum decision 129's ruling that per-provider corruption coverage beyond import refusal satisfies it by design; "upgrade and corruption scenarios are documented" is now met — docs/operations/recovery.md is the cross-provider page bullet 5 needed, with the filesystem section as its substantial half. Of the nine bullets above, the EF Core migrations bullet, the API baseline bullet, bullet 1, bullet 2, bullet 3, bullet 4, bullet 5, bullet 6, and bullet 8 are fully shipped. Every ROADMAP 0.2 bullet has shipped. The 0.2 milestone is complete, as of 2026-09-15.

Exit criteria: all built-in providers pass one conformance suite — met: addendum decision 129 (Phase 22) declares this met on the ruling that per-provider corruption coverage beyond import refusal satisfies it by design, consistent with the standing decisions behind Phase 16 decision 63 and addendum decision 98, rather than by building a shared corruption-conformance suite; upgrade and corruption scenarios are documented; CI builds every supported target framework — CI builds net8.0, net9.0 and net10.0 for the thirteen cross-targeting packages on three operating systems and tests net10.0 only, because tests/Directory.Build.props:6 is the only place any test project declares a framework, so no behaviour is ever tested on net8.0 or net9.0; a net8.0/net9.0 test-run job is a recorded Phase 23 candidate.

0.3: durable distributed coordination

Distributed behavior will be explicit and capability-based.

  • IStateChangeFeed with durable cursors and resumable history projection
  • provider-native notifications as an optimization over ledger cursors
  • leases for interval loaders and singleton maintenance workers
  • partition discovery for stores that can support it efficiently
  • distributed coherent capture capability with a documented consistency level
  • import/export and restore tooling with fingerprint and schema validation
  • replication lag metadata and read policies for tiered or replicated deployments
  • outbox-oriented bridges for message brokers without making observation a broker

Exit criteria: no distributed method silently falls back to process-local semantics; capability discovery tells callers exactly which guarantee is available.

0.4: state graph and orchestration

Statesman can grow from isolated authorities into an inspectable state graph without becoming a workflow engine.

  • declared selectors and derived state with dependency manifests
  • cycle detection and deterministic invalidation propagation
  • materialized versus ephemeral derived-state policies
  • typed effects that execute after accepted transitions with idempotency metadata
  • batch declarations and generated strongly typed accessors
  • source-generated manifests for Native AOT and startup-sensitive applications
  • Blazor, WPF, WinUI, MAUI, and reactive binding adapters built over the same observation contracts

Exit criteria: derived behavior appears in the manifest, can be visualized, and cannot create hidden writes or dependency cycles.

1.0: stable authority contract

A 1.0 release requires:

  • a reviewed and compatibility-baselined public API
  • conformance coverage for every first-party provider
  • documented upgrade, backup, restore, and schema migration procedures
  • stable manifest canonicalization and versioning rules
  • performance baselines for high-cardinality partitions and long histories
  • security review of inspection endpoints, provider inputs, and serialization boundaries
  • analyzer false-positive and suppression guidance validated against real migrations
  • at least one production adoption that exercises multi-source loading, durable history, and rolling upgrades

Deliberate non-goals

The roadmap does not turn Statesman into a general database, object-relational mapper, arbitrary query engine, distributed transaction coordinator, message broker, or event-sourcing framework. Integrations with those systems remain adapters around the state authority. Any proposal that weakens this boundary must explain why it cannot be an optional capability or external projection.

Edit this page