> > > SpecFactSimulatorArchitectureSpec

Fact Simulator Architecture Spec

Purpose

Define the internal architecture, parser constraints, canonicalization rules, and testing criteria for the SOTER Fact Simulator and Cryptographic Ledger tool (soter_sim). This specification targets implementers of the tool; users and operators should instead follow CookbookSimulatorUser.md and CookbookSimulatorDev.md.

Scope

The three architectural components of soter_sim: Fact Generator, Thin Ledger, and Providence Verifier.
The target soter_sim package/repository structure.
Technology and dependency choices, including the dual-mode (SQLite + flat file) ledger architecture.
The canonicalization pipeline used to compute deterministic fact and ledger-entry hashes.
Anomaly injector logic and the full set of injectable anomaly types.
Verifier integrity rules (the audit checks run by soter-verify).
Developer test suite requirements.

Non-Goals

User-facing CLI workflow and scenario authoring instructions — see CookbookSimulatorUser.md.
Developer setup/build/test-run instructions — see CookbookSimulatorDev.md.
The Logos-language .fact block syntax — see LogosFactFileSpec.md.
Akasha's protocol-level fact envelope and production backend strategy — see AkashaLedgerTechnicalSpec.md.
Analytical ledger partitioning and workflow board projections — see AnalyticalLedgerWorkflowSpec.md.

Terminology

Term	Meaning
Fact Generator (`soter simulator`)	Generates fact sequences from Logos scenario templates, with a deterministic anomaly injector.
Thin Ledger (`soter-ledger`)	Records facts in an append-only file ledger; separates raw payloads (`.fact`) from cryptographic audit blocks (`.ledger`).
Providence Verifier (`soter-verify`)	Deterministically analyzes ledger hash integrity, sequence continuity, and state-transition conformance.
Canonicalization (c14n)	Deterministic serialization of an AST to bytes, independent of cosmetic formatting.
Anomaly injector	The `anomaly_injector.py` module that deterministically introduces scenario violations at configured rates.
Merkle checkpoint	A recomputable Merkle root over a range of ledger entries, used for tamper-evident batch verification.

Model or Contract

Architectural Component Overview

[Logos Scenario (.scenario)] ──> [Generator] ──> [Anomaly Injector] ──> [Payload (.fact)]
                                                                               │
[Verifier] <── [Ledger Index (.ledger)] <── [Ledger Writer] <──────────────────┘

Target Repository Structure

soter_sim/
  pyproject.toml        # types, dependencies (Pydantic, Typer, Lark, pytest)
  README.md             # Developer setup, testing, and operation index

  src/
    soter_sim/
      __init__.py
      cli.py            # Typer CLI application entry point
      config.py         # Workspace path resolvers

      logos/            # Logos DSL parser and c14n engine
        __init__.py
        ast.py          # AST Node dataclasses
        parser.py       # Strict Logos parser/tokenizer
        serializer.py   # Pretty printer for AST -> Logos string
        canonical.py    # Logos Canonicalization (c14n) implementation

      ledger/           # Cryptographic Ledger and Ledger Log management
        __init__.py
        writer.py       # Append-only Fact & Ledger entry creator
        reader.py       # Ledger crawler and reader
        hashchain.py    # Blake3 / SHA256 chain validators
        merkle.py       # Merkle tree root and checkpoint generators
        manifest.py     # manifest.ledger handler
        verifier.py     # Verification runner

      simulation/       # Business scenario execution engines
        __init__.py
        scenario.py     # Logos scenario compiler and validator
        organization.py # Org state spaces and actor registries
        generator.py    # Process step executor
        anomaly_injector.py # Anomaly injection logic
        clock.py        # Simulated virtual clocks
        ids.py          # Bitemporal ID generator (f-..., a-...)

      analysis/         # Audit report and metrics generators
        __init__.py
        conformance.py  # Conformance checks against normal_flow
        anomaly_report.py # JSON LLM-digest report builder

  scenarios/            # Logos scenario configurations (Recruitment, Administrative, Warehouse)
    jobbot_recruitment.scenario
    municipality_decision.scenario
    warehouse_order.scenario

  tests/                # Complete Pytest verification suite
    test_canonical_hash.py
    test_hashchain.py
    test_merkle.py
    test_generate_jobbot.py
    test_verify_tamper.py

Technology & Dependencies

Language Version: Python 3.12+
Validation & CLI: pydantic (V2 preferred for AST validation), typer (for CLI subcommands), lark (for Logos scenario parser).
Hashing Engine: hashlib.sha256 or blake3 (if compiling blake3 is troublesome locally, standard hashlib.sha256 must be used as a fallback).
Testing: pytest
Ledger Architecture (Dual-Mode SQLite & FS):
Operational DB: For production performance, indexing speed, and scale, the active ledger engine implements an SQLite-backed relational transactional database as its high-performance ledger engine.
Debugging & Educational FS Option: The system MUST retain full synchronization with plain-text .fact and .ledger files. The file-system-based append-only file structures (stream/, facts/, checkpoints/) must be maintained as a dual option. This provides a transparent, zero-dependency environment for educational exercises, visual troubleshooting, and manual auditing.
Verification Target: The Providence verifier (soter auditor) must be capable of auditing both the high-performance SQLite database indexes and the visual .ledger flat file streams.

Rules

Cosmetic formatting immunity: Spacing, comments, or indentation changes must never break the cryptographic signature of a fact or ledger entry.
When hashing a .ledger (LedgerEntry) block, the parser must completely omit the hash field itself during canonical serialization.
The anomaly injector must be deterministic and rate-controlled; the same seed and scenario must reproduce the same anomaly placements.
The verifier must run all applicable integrity checks and report violations rather than halting at the first failure, so a single run yields a complete audit.

Canonicalization Pipeline

Parse: Tokenize the Logos syntax (.fact or .ledger) into an AST tree representation.
Sort: Arrange AST element attributes alphabetically by key.
Format: Serialize the sorted AST back into a deterministic, compact byte stream (UTF-8) with zero extra spaces or newline variances.
Exclude hash key: When hashing a .ledger (LedgerEntry) block, the parser must completely omit the hash field itself during canonical serialization.

fact_hash = SHA-256(canonical_bytes(AST(fact_file)))
entry_hash = SHA-256(canonical_bytes(AST(ledger_file \ {hash})))

Anomaly Injector Logic

The anomaly_injector.py module must implement a deterministic, rate-controlled process transformation engine:

missing_event: Skips a normal flow event (e.g. skips CVParsed and goes CV -> score).
duplicate_event: Appends an identical fact step within the same case.
out_of_order_event: Swaps the chronological sequence of two adjacent events.
wrong_actor: Assigns an action to an actor ID not participating in the case sequence.
late_event: Simulates clock drift or network lag, pushing recorded_at significantly past occurred_at.
tamper_attempt: Manually edits an established fact payload string on disk without rewriting the hashchain.
hashchain_break: Modifies a ledger hash pointer or sequence numbering.

Other advanced violations (wrong_role, semantic_violation, mass_balance_violation) should be prepared in the enum as stubs with TODO markers.

Simulation Logic and Confrontation Mechanics

The simulator runs in two distinct execution modes:

Deterministic Simulation: Replays the scenario steps in exact line-order Sequence, ensuring a predictable, reproducible series of events.
Probabilistic Simulation: Leverages decision branching weights on gateways and randomized timing delays (within defined min/max bounds) to generate varied process trajectories.

Confrontation Modes

StrictConfrontation: Checks each observed event sequence against the reference normal_flow exactly. Any missing, extra, or out-of-order event is immediately flagged as a high-severity violation.
StatisticalConfrontation: Rather than enforcing exact step-by-step conformance, validation is performed at the macro level. It compares the overall distribution, frequency, and transition rates of events against historical bounds or thresholds using confidence intervals (statistical tests).

Parser Compilation

The parser compiles .scenario files directly into Soter AST using a Lark-based grammar compiler. This completely eliminates dependency on pyyaml for scenario configurations.

Verifier Integrity Rules

The Providence verifier (verifier.py) must deterministically run an audit on the target ledger stream, covering:

Sequence Continuity: Check that ledger sequence numbers (seq) form a continuous, gapless range (1, 2, 3, ...).
Hashchain Integrity: Check that each entry's prev_hash is identical to the canonical hash of the preceding entry.
Fact Authenticity: Recalculate canonical hash of every .fact file on disk and compare it to the fact_hash pointer in the corresponding .ledger block.
Merkle Validation: Recompute Merkle root trees for established checkpoints and verify them against recorded merkle_root values.
Flow Conformance: Match each case flow sequence against the declared normal_flow list, identifying missing or inserted steps.
Orphans: Detect facts recorded in the ledger containing case references that do not exist or have no normal starting boundaries.
Temporal Chronology: Enforce bitemporal alignment: recorded_at >= occurred_at always.
Delay Threshold: Detect registration latency anomalies (recorded_at - occurred_at > max_lag).

Examples

See CookbookSimulatorUser.md for the .fact and .ledger file format examples and CLI invocation examples used against this architecture.

Validation

Developers must deliver a green pytest suite satisfying these test criteria:

`test_canonical_hash.py`

Verify that altering spacing, indentation, or adding comments inside a .fact text file does not alter its calculated canonical hash.
Verify that changing a value string inside a field modifies the hash.
Verify that the technical hash attribute does not influence .ledger canonical bytes.

`test_hashchain.py`

Verify that a clean, untampered ledger stream passes all verification checks.
Verify that manually modifying a single fact value in the middle of the chain breaks the subsequent ledger prev_hash comparison.
Verify that deleting an entry breaks sequence continuity.

`test_merkle.py`

Verify that Merkle tree generation is deterministic and consistent.
Verify that altering a single fact file changes the resulting checkpoint root.

`test_verify_tamper.py`

Verify that manual byte modification of a .fact payload file on disk immediately triggers a high-severity tamper violation.

Compatibility

This is an implementation-facing specification for the standalone soter_sim tool. It does not define a stable external protocol; changes to package/module layout are expected as the tool matures, provided the CLI surface and file formats documented in CookbookSimulatorUser.md remain stable. The dual-mode ledger and hash formula summarized in CookbookSimulatorDev.md are the developer-facing digest of the canonicalization and hashchain rules defined in full here.

Open Questions

Should the hashchain_break and advanced-violation stubs (wrong_role, semantic_violation, mass_balance_violation) be promoted out of TODO status, and if so, on what schedule?
Should soter_sim eventually consume Akasha's canonical fact envelope directly instead of maintaining its own .fact/.ledger file pair?