Business Index SOSA Current State and Gap Analysis

Date: 2026-07-20

Governing sources

This analysis uses the Business Index SOSA implementation brief and the current SOSA repository as separate authorities. SOSA defines the architecture; this repository defines one product implementation. The current SOSA hierarchy is:

BDH -> SOSA -> SOM -> SI -> implementations

SOSA, SO, SOM, and SI are draft specifications. Their current documents define the layer boundaries and required concerns but intentionally leave several field-level questions open. The Business Index brief supplies the product's minimum operational requirements without redefining SOSA.

Legacy SOSI material was also inspected for continuity. Its detailed guidance on identifiers, evidence, relationships, confidence, governance, and versioning is treated as informative historical material. Current code and documentation use SOSA, SOM, and SI terminology.

Verified current implementation

The existing local MVP is a conventional document-retrieval system:

1. Approved folders are mounted read-only and reconciled periodically.
2. Supported files are hashed and extracted locally.
3. Extracted text is divided into overlapping chunks.
4. Ollama creates local embeddings for those chunks.
5. PostgreSQL and pgvector store source records, chunks, and vectors.
6. Search embeds a question and returns the nearest chunks by cosine distance.
7. A local chat model answers from those chunks and must emit source markers.

The database has semantic_objects, object_evidence, and object_relationships tables, but ingestion currently creates only one document object per file. Re-indexing deletes and recreates that object and its chunk evidence. Relationships are not extracted or retrieved. Search is vector-only. There is no durable cross-source identity, immutable object version, evidence-span model, auditable correction workflow, or temporal graph retrieval.

The existing behaviour is therefore an honest chunk-based vector baseline. It must remain available for evaluation, but it is not the SOSA implementation.

Gap against the required invariants

Invariant	Current state	Required change
Meaning first	Chunks and document objects dominate	Persist approved business concepts as governed Semantic Objects
Persistent identity	Object identity is recreated with a file	Resolve aliases across sources into durable object identities
Evidence and provenance	Document objects point to disposable chunks	Preserve immutable source versions and character-addressed evidence spans
Explicit relationships	Table exists but has no pipeline or retrieval role	Persist typed, directional, evidence-backed relationships
Time and change	Current extracted text overwrites prior text	Add object/source versions, validity time, learned time, and supersession
Governed adaptation	No feedback history	Record confirm, reject, correct, merge, split, and supersede events immutably
Model independence	Chunk vectors can be rebuilt, but object state is absent	Keep object identity separate from model-specific representations
Local-first privacy	Local Ollama and read-only mounts are implemented	Preserve this boundary for semantic extraction and diagnostics
Hybrid retrieval	Vector chunks only	Combine lexical, vector, object, relationship, and temporal signals

Smallest coherent vertical slice

The first implementation cycle will:

• add non-destructive SOSA core tables and migration guidance;
• support Client, Project, Decision, Commitment, and Problem objects;
• normalize local-model output at an explicit extraction boundary;
• resolve identity from canonical labels and aliases while recording uncertainty;
• preserve immutable source versions, evidence spans, object versions, aliases, relationships, extraction runs, representations, and feedback events;
• stop deleting governed meaning when a source is re-indexed;
• add hybrid search that can return source facts, objects, and relationships with their evidence;
• add focused invariant and evaluation tests;
• retain the existing vector-chunk path as the conventional baseline.

This cycle will not add new integrations, autonomous actions, a broad object ontology, or cloud processing. OCR remains an explicit gap unless implemented and verified separately.

Principal risks

• Local model extraction can be incomplete or inconsistent. Candidate results must stay unreviewed and evidence-bound rather than being promoted to fact.
• Alias matching can conflate distinct clients or projects. Ambiguous matches must be recorded for review instead of merged automatically.
• The current SOM specification is still draft. Product-specific fields must not be described as new SOSA requirements.
• PostgreSQL migration and rollback must preserve the already indexed local corpus and must not depend on embedding dimensions.