Analytical Ledger & Workflow Projection Spec

Purpose

Define the technical specification for the SOTER-native, event-sourced workflow system retrofitted over the analytical ledger (a-ledger): the physical partitioning of analytical runs, the append-only event/fact formats they write, the projection manifest, the write/read paths, and the service abstractions that implement them.

The user-facing behavior this specification underlies — reading run parameters, dragging cards between lanes, and using run controls — is documented for analysts in ManualAnalyst.md; this document is its technical/architectural counterpart.

Scope

• Physical, tenant/world/run partitioning of the analytical ledger on disk.
• The run_started, card_created, card_moved, and run_stopped fact formats.
• The projection manifest format for compiled read-only projections (SQLite / JSON).
• The write path (UI mutation to appended fact) and read path (board request to replayed or cached projection).
• Architectural membrane rules separating analytical and operational ledger streams.
• The AnalyticalLedger, WorkflowRuntime, and WorkflowReducer service abstractions.
• Non-goals for the analytical ledger / workflow projection MVP.

Non-Goals

• User-facing instructions for operating the workflow board (New Run, Close Run, Replay, drag-and-drop) — see ManualAnalyst.md.
• The Logos-language .fact block syntax used by the operational ledger — see LogosFactFileSpec.md.
• Akasha's protocol-level fact envelope, hash-chain, and storage backend strategy for the operational ledger — see AkashaLedgerTechnicalSpec.md. The analytical ledger described here is a separate, sandboxed stream (see Architectural Membrane Rules) and does not currently share Akasha's backend.
• Full BI or multi-dimensional analytics.
• Process Token lifecycle simulations.
• Advanced custom multi-graph visualization widgets on cards.
• Automatic promotion of analytical sandbox facts directly into the operational ledger.
• Conflict resolution for multiple simultaneous users writing to the same simulation run.

Terminology

Model or Contract

Physical Partitioning of the a-Ledger

The analytical ledger is logically tenant-scoped, but physically partitioned by world and run to prevent cold starts and allow targeted history replays.

soter-tenants/{tenant}/
  tenant.json
  .gitignore
  ledgers/
    production/
      ledger.sqlite3
      ledger.manifest
    analytical/
      {world_id}/
        {run_id}/
          run.scenario
          ledger.fact
          output/
            metadata/
            databases/
            dashboards/
            logs/

Event/Fact Specifications (YAML)

All workflow state transitions are recorded as append-only facts in the local ledger.fact stream.

run_started

Written when a new analytical simulation run is initialized.

fact: u-fact-000001
realm: analytical
type: run_started
tenant: acme
world: sales-simulation
run: run-2026-05-22-001
model_refs:
  - input/models/sales.model
  - input/models/pricing.model
started_by: user1
started_at: 2026-05-22T20:26:00Z

card_created

Written when an analyst instantiates a new ActionOccurrence (card) on the board.

fact: u-fact-000002
realm: analytical
type: card_created
tenant: acme
world: sales-simulation
run: run-2026-05-22-001
card: u-card-prepare-offer-101
lane: planned
position: 0
subject_ref: SalesEmployee
action_ref: PrepareOffer
object_ref: Offer_123
title: "SalesEmployee -> PrepareOffer -> Offer_123"
actor: user1
recorded_at: 2026-05-22T20:28:00Z

card_moved

Written when a card's lane or position order changes.

fact: u-fact-000045
realm: analytical
type: card_moved
tenant: acme
world: sales-simulation
run: run-2026-05-22-001
actor: user1
card: u-card-prepare-offer-101
from_lane: planned
to_lane: executed
position: 1
recorded_at: 2026-05-22T20:30:00Z

run_stopped

Written when the active simulation run is finalized.

fact: u-fact-000180
realm: analytical
type: run_stopped
tenant: acme
world: sales-simulation
run: run-2026-05-22-001
head_hash: "a9bf28..."
stopped_by: user1
stopped_at: 2026-05-22T21:00:00Z

Projection Manifest Specification

When compiling a persistent, read-only projection (e.g., SQLite or JSON cache), SOTER writes a manifest confirming the source metadata.

projection: workflows-board
id: u-proj-20260522-001
realm: analytical
ledger: ledger/analytical
tenant: acme
world: sales-simulation
run: run-2026-05-22-001
source:
  ledger_file: workspace/acme/ledger/analytical/worlds/sales-simulation/runs/run-2026-05-22-001/ledger.fact
  from_seq: 1
  to_seq: 180
  head_hash: "a9bf28..."
outputs:
  sqlite: projections/workflows.sqlite3
  json: projections/workflows.board.json
mode: read_only
projection_version: 1
generated_at: 2026-05-22T21:05:00Z

Service / API Abstractions

AnalyticalLedger

Manages the raw file append operations and reads.

class AnalyticalLedger:
    def append(self, tenant_id: str, world_id: str, run_id: str, event_data: dict) -> None:
        """Appends a new serialized YAML block to the run's ledger file."""
        pass

    def read_run(self, tenant_id: str, world_id: str, run_id: str) -> list[dict]:
        """Reads and parses all events from the physical run ledger file."""
        pass

WorkflowRuntime

Handles the high-level semantic actions initiated by users or simulation runners.

class WorkflowRuntime:
    def get_or_create_active_run(self, tenant_id: str, world_id: str, model_refs: list[str]) -> str:
        """Finds the latest open run or creates one, appending run_started."""
        pass

    def create_card(self, tenant_id: str, world_id: str, run_id: str, card_data: dict) -> None:
        """Appends card_created fact."""
        pass

    def move_card(self, tenant_id: str, world_id: str, run_id: str, card_id: str, to_lane: str, position: int) -> None:
        """Appends card_moved fact."""
        pass

WorkflowReducer

Replays a stream of run events to reconstruct the board state.

class WorkflowReducer:
    def reduce(self, events: list[dict]) -> dict:
        """Aggregates an ordered sequence of events into a dict of lanes and cards."""
        pass

Rules

• Projection Disposability: Deleting any SQLite database or JSON projection cache must never lose truth. The true state remains fully replayable from ledger.fact.
• Strict Segregation: Analytical run event streams must never mix with operational ledger event streams.
• No Direct SQLite Mutations: The kanban.db or individual workspace workflows.sqlite3 databases must be treated as compiled read-only views. No direct INSERT or UPDATE queries are allowed on them during runtime, except by the projection compiler/reducer during replay.

Examples

Write Path

1. The user performs a drag-and-drop or visual mutation in the UI.
2. The UI sends a command payload to the WorkflowRuntime.
3. WorkflowRuntime serializes a new fact block and appends it to {run_id}/ledger.fact.
4. The projection cache is marked stale or immediately updated in memory.
5. The view is re-rendered from the new projection state.

Read Path

1. The UI requests the board for a specific (tenant, world, run).
2. The system checks if a fresh projection snapshot exists on disk.
3. If a snapshot is present, the system loads it directly.
4. If no snapshot is present (or is marked stale), the WorkflowReducer reads {run_id}/ledger.fact from sequence 1, replays all facts in-memory, and returns the computed visual state.

This matches the simplified appended-event / reducer / projection flow described for analysts in ManualAnalyst.md.

Validation

• A run's board state, when reconstructed via WorkflowReducer.reduce from sequence 1, must be identical to the state produced by any cached projection for the same to_seq.
• Deleting a projection (SQLite or JSON) and triggering Replay must reproduce byte-for-byte equivalent board state.
• No test or runtime code path may issue direct INSERT/UPDATE statements against a compiled projection database outside the projection compiler/reducer.

Compatibility

This specification describes the current MVP scope of the analytical ledger and workflow projection system. It is intentionally decoupled from the operational Akasha ledger (see Non-Goals); a future revision may define a promotion path from analytical sandbox facts to operational facts, but that is explicitly out of scope for the MVP (see Non-Goals).

Open Questions

• Should AnalyticalLedger eventually share Akasha's backend abstraction (file+SQLite, PostgreSQL, ...), or remain a separate, simpler implementation given its sandboxed nature?
• What is the promotion path, if any, for analytical facts that a user wants to commit as operational truth?
• How should conflict resolution work if this MVP's single-writer assumption is later relaxed?

Related Documents

• ManualAnalyst.md - analyst-facing workflow board operation (run controls, drag-and-drop).
• LogosFactFileSpec.md - Logos-language .fact block syntax used by the operational ledger.
• AkashaLedgerTechnicalSpec.md - protocol/storage layer for the operational ledger.
• AkashaLedgerOverview.md - entry point for ledger documentation.
• FactSimulatorArchitectureSpec.md - related fact-generation and ledger-verification architecture.
• DocumentationStyleGuide.md - writing profile and document-type rules.
• TechnicalSpecTemplate.md - technical document template.