68 lines
2.1 KiB
Markdown
68 lines
2.1 KiB
Markdown
|
# ADR-011: Four-Layer Data Architecture
|
||
|
|
|
||
|
|
## Status
|
||
|
|
Accepted
|
||
|
|
|
||
|
|
## Context
|
||
|
|
We need a clear separation between raw data, extracted knowledge, and decision intelligence to avoid building expensive, tightly-coupled AI systems.
|
||
|
|
|
||
|
|
## Decision
|
||
|
|
Separate into four layers:
|
||
|
|
|
||
|
|
### Layer 1: Raw Archive (cheap, immutable)
|
||
|
|
- Original files never changed
|
||
|
|
- Hash guarantees integrity
|
||
|
|
- Storage policy: active (30 days) → archive (long-term, cheaper)
|
||
|
|
|
||
|
|
### Layer 2: Knowledge Extraction
|
||
|
|
- Objects, geometry, GPS, classifications, bounding boxes, embeddings
|
||
|
|
- Observations and Evidence created
|
||
|
|
- Structured knowledge used for most queries, not video files
|
||
|
|
|
||
|
|
### Layer 3: Knowledge Graph / Ontology
|
||
|
|
- Road A12 → has Observation → Crack → belongs to Area → Stockholm
|
||
|
|
- Answers: "Show all main roads where cracking increased last 12 months"
|
||
|
|
- Comes from knowledge graph, not by reading video files
|
||
|
|
|
||
|
|
### Layer 4: Decision Intelligence
|
||
|
|
- Verified Decision Cases
|
||
|
|
- Curated datasets for AI training
|
||
|
|
- Business impact tracking
|
||
|
|
|
||
|
|
## Key Principles
|
||
|
|
|
||
|
|
1. **Archive Artifact** = Original file, never modified
|
||
|
|
2. **Knowledge Artifact** = Extracted knowledge, can be re-generated
|
||
|
|
3. **Ontology before model** — taxonomy answers "what does it mean in our domain?"
|
||
|
|
4. **AI models trained on curated datasets**, not whole archive
|
||
|
|
|
||
|
|
## Data Lifecycle
|
||
|
|
|
||
|
|
```
|
||
|
|
Upload → Immutable Archive → Metadata Extraction → Knowledge Extraction
|
||
|
|
→ Ontology Mapping → Decision Pipeline → Learning → Archive Retention
|
||
|
|
```
|
||
|
|
|
||
|
|
Each step produces a new Artifact with version history.
|
||
|
|
|
||
|
|
## Long-term Goal
|
||
|
|
|
||
|
|
"Every real observation is converted once to structured knowledge and can then be reused infinitely for analysis, decision support, history, training, and future AI models."
|
||
|
|
|
||
|
|
## Consequences
|
||
|
|
|
||
|
|
### Positive
|
||
|
|
- Raw data = revisionable archive
|
||
|
|
- Real product = growing knowledge base + verified Decision Cases
|
||
|
|
- Can swap AI models without losing originals
|
||
|
|
- Cheaper storage (archive vs active)
|
||
|
|
|
||
|
|
### Negative
|
||
|
|
- More complex pipeline
|
||
|
|
- Need to manage extraction models
|
||
|
|
- Knowledge graph maintenance
|
||
|
|
|
||
|
|
## Related
|
||
|
|
- ADR-002: Artifact is common base contract
|
||
|
|
- ADR-003: Event Sourcing for traceability
|