Infrastructure

Every Sixb runtime is wired to five infrastructure providers. All are required and passed to createSixb(). They split into three storage slots and two messaging slots.

A typical local setup uses durable on-disk storage and in-memory messaging:

import { createSixb, InMemoryBroker, InMemoryQueues } from "@sixb/core"
import { SqliteStorage } from "@sixb/sqlite"
import { LocalLakeStorage } from "@sixb/lake-local"
import { LocalBlobStorage } from "@sixb/blob-local"

export const sixb = await createSixb({
  id: "northline",
  storage: new SqliteStorage({ path: ".sixb" }),
  lakeStorage: new LocalLakeStorage({ path: ".sixb/lake" }),
  blobStorage: new LocalBlobStorage({ basePath: ".sixb" }),
  broker: new InMemoryBroker(),
  queues: new InMemoryQueues(),
})

createSixb() is async — always await it.

The three storage slots

Sixb separates storage by access pattern. The slots are not interchangeable, and each takes its own provider.

• storage — the operational store. Objects and their properties, links, appended telemetry, and run-history tables for actions, syncs, pipelines, projections, and workflows. This is the database behind sixb.objects(...) reads and writes.
• lakeStorage — the versioned data lake. Holds datasets produced by syncs, pipelines, and connectors, with snapshots and version compatibility.
• blobStorage — content-addressed binary blobs. When a property or dataset column is a fileRef, the bytes live here and the other stores keep only the reference.

Broker vs queues

The two messaging slots are not the same thing — keep them distinct.

For ontology facts, the operational database is authoritative: the Materializer writes ontology_commits and ontology_outbox atomically before best-effort broker publication. The broker is the retained delivery/read surface, while queues turn requested work into running work with leases and retries. The queues provider exposes one lane per kind of background work:

sixb.queues.actions
sixb.queues.syncRuns
sixb.queues.pipelines
sixb.queues.projections
sixb.queues.workflows

Storage providers must preserve bounded outbox claims, lease-fenced settlement, retry summaries, published-row retention, and child-first cleanup of terminal source materializations. Pending rows, nonterminal sources, and ontology_commits are never removed by age.

ObjectStorage and TimeseriesStorage are read models. Actions, runtime CRUD, projections, and telemetry all write through the Materializer and its private OntologyStorage.materializations protocol. Providers must not expose an event-to-row writer or interpret domain events as storage commands.

The required storage.ping() readiness check must be lightweight and read-only. It must not open a write transaction, run migrations, or acquire a migration/advisory lock. Schema validation is a separate cached check and retries failures with a cooldown.

Storage schema boundary

The initial SQLite and PostgreSQL schemas are the only supported ontology schema. They intentionally contain no compatibility importer or upgrade path from earlier unpublished schemas. Before switching an existing environment:

freeze writers and drain jobs
  -> export retained project-owned data
  -> create fresh Sixb storage
  -> run normal syncs and replacement projections
  -> replay source-less state through Actions or runtime CRUD
  -> verify, then switch configuration

Project-specific mappings and migration scripts stay outside the framework.

Retention

The API role purges expired rows every 60 seconds, in the same maintenance pass that catches the outbox up.

Pending outbox rows and nonterminal sources are live data and are never purged by age. Size the disk with ontology_commits in mind: the pre-0.1 line has no purge for it.

export const sixb = await createSixb({
  // ...
  ontologyMaintenance: {
    intervalMs: 60_000,
    publishedOutboxRetentionMs: 24 * 60 * 60_000,
    terminalSourceRetentionMs: 24 * 60 * 60_000,
    cleanupLimit: 1_000, // rows deleted per table per pass
  },
})

Provider matrix

Pick a real provider class for each slot. InMemory* providers come from @sixb/core and need no extra install — they are for development and tests only, never production.

logger is the one optional slot. Omit it for broker-only logging (still readable in Atlas, sixb.logs, and the client logs builder); add a LoggerProvider such as PinoLogger to also emit process-level output. See Logging.

Production example

A durable multi-process setup pairs PostgreSQL, DuckLake, S3 blobs, and Redis-backed messaging:

import { createSixb } from "@sixb/core"
import { PostgresStorage } from "@sixb/pg"
import { DuckLakeStorage } from "@sixb/ducklake"
import { S3BlobStorage } from "@sixb/blob-s3"
import { RedisBroker } from "@sixb/broker-redis"
import { BullMqQueues } from "@sixb/queues-bullmq"

export const sixb = await createSixb({
  id: "acme-corp",
  storage: new PostgresStorage({ connectionString: process.env.DATABASE_URL! }),
  lakeStorage: new DuckLakeStorage({
    catalog: { type: "postgres", host: "localhost", database: "lake", user: "sixb", password: "secret" },
    dataPath: "s3://acme-lake/data",
  }),
  blobStorage: new S3BlobStorage({ bucket: "acme-lake", region: "us-east-1", basePath: "sixb" }),
  broker: new RedisBroker({ connection: { url: "redis://localhost:6379" } }),
  queues: new BullMqQueues({ connection: "redis://localhost:6379" }),
})

See Deployment for running this in production.

Migrations

SQL-backed storage providers (@sixb/pg, @sixb/sqlite) own their schema and ship migrations. The six roles that touch the schema apply them at startup, so the explicit command is for running the migration as its own deploy stage:

sixb db migrate

This loads your runtime and applies pending migrations against the configured storage provider. In-memory and file-lake providers have no schema and skip this step. See Deployment for which roles migrate and how to start them without migrating.

Related

• Runtime — createSixb() and convention-based discovery
• Events — the domain events the broker carries
• Deployment — running a durable setup in production