Bernt
9a817ac82a
- Added architecture diagram: - Developer UI / Training API / Experiment API - Event Bus / Queue - Storage Layer (S3/R2 + PostgreSQL + Neo4j) - Processing Pipelines - Seven separate pipelines (not one big AI loop): 1. Ingestion — raw images/video/GPS/EXIF → Raw Dataset 2. Dataset — sort, deduplicate, quality control → Validated Dataset 3. Annotation — AI suggestions, manual correction → Verified Dataset 4. Training — hyperparameters, checkpoints, GPU → Model Artifact 5. Evaluation — precision, recall, F1, decision accuracy → Report 6. Replay — compare model versions → Regression Report 7. Decision Validation — full chain → Validation Report - Bucket structure: raw-images, raw-video, missions, datasets, annotations, training, models, replays, evaluation, decision-cases, field-trials, exports, archives - AI Job Queue: all jobs asynchronous (Upload → Queue → Worker → GPU → Storage → Notification) - Core principles updated: - All artifacts immutable and versioned - Every change traceable to experiment, model, dataset, decision - Dashboard shows: Datasets (Healthy/Needs Review/Corrupted), Training Jobs, Decision Cases, Replay Jobs - Decision Cases emphasized as most important asset Rationale: Separate pipelines make system easier to debug, improve, and swap components. Immutability aligns with E-001 Git/traceability principles. Decision Cases become the unique asset over time.
367 lines
8.1 KiB
Markdown
367 lines
8.1 KiB
Markdown
# LANDVEX INTELLIGENCE LAB
|
|
|
|
**Internal Development Environment for Control Intelligence**
|
|
|
|
| | |
|
|
|---|---|
|
|
| **Version** | 1.0 |
|
|
| **Status** | SPECIFICATION |
|
|
| **Purpose** | Build, test, and verify LandveX Control Intelligence before production |
|
|
|
|
---
|
|
|
|
## Core Principles
|
|
|
|
**LandveX Intelligence Lab does not produce AI models. It produces verified Control Intelligence.**
|
|
|
|
This is an internal tool. Never a customer product.
|
|
|
|
**All artifacts are immutable and versioned.**
|
|
- Raw images are never modified
|
|
- Annotations are versioned
|
|
- Models are versioned
|
|
- Evaluation reports are versioned
|
|
- Decision Cases are versioned
|
|
- Replay results are saved as new artifacts
|
|
|
|
**Every change must be traceable to a specific experiment, model version, dataset, and decision.**
|
|
|
|
---
|
|
|
|
## Repository
|
|
|
|
```
|
|
landvex-intelligence-lab
|
|
```
|
|
|
|
Separate from:
|
|
- `quixzoom-app`
|
|
- `landvex-web`
|
|
- `aamos-core`
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
LandveX Intelligence Lab
|
|
│
|
|
┌─────────────────┼─────────────────┐
|
|
│ │ │
|
|
Developer UI Training API Experiment API
|
|
│ │ │
|
|
└─────────────────┴─────────────────┘
|
|
│
|
|
Event Bus / Queue
|
|
│
|
|
─────────────────────────────────────────────────────
|
|
Storage Layer
|
|
Images │ Videos │ Missions │ Models │ Logs
|
|
S3/R2 Buckets + PostgreSQL + Neo4j
|
|
─────────────────────────────────────────────────────
|
|
│
|
|
Processing Pipelines
|
|
```
|
|
|
|
## Navigation
|
|
|
|
```
|
|
Dashboard
|
|
├── Models
|
|
├── Datasets
|
|
├── Annotations
|
|
├── Training
|
|
├── Evaluation
|
|
├── Decision Cases
|
|
├── Replay
|
|
├── Validation
|
|
├── Promote Model
|
|
└── Settings
|
|
```
|
|
|
|
---
|
|
|
|
## Dashboard
|
|
|
|
Shows AI status, not business data.
|
|
|
|
```
|
|
┌─────────────────────────────────────┐
|
|
│ Datasets │
|
|
│ Healthy: 12 │
|
|
│ Needs Review: 3 │
|
|
│ Corrupted: 0 │
|
|
├─────────────────────────────────────┤
|
|
│ Training Jobs │
|
|
│ Running: 2 │
|
|
│ Queued: 1 │
|
|
│ Completed: 47 │
|
|
│ Failed: 0 │
|
|
├─────────────────────────────────────┤
|
|
│ Decision Cases │
|
|
│ Validated: 23 │
|
|
│ Pending: 5 │
|
|
│ Rejected: 2 │
|
|
├─────────────────────────────────────┤
|
|
│ Replay Jobs │
|
|
│ Ready: 8 │
|
|
│ Running: 1 │
|
|
│ Finished: 34 │
|
|
└─────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Mission Replay
|
|
|
|
Click through the entire chain:
|
|
|
|
```
|
|
Video → Frame → Bounding boxes → Detected objects → Evidence → Finding → Decision → Business Impact
|
|
```
|
|
|
|
---
|
|
|
|
## Annotation
|
|
|
|
```
|
|
┌─────────┬─────────────┬──────────────┐
|
|
│ Video │ AI Suggestion│ Manual │
|
|
│ │ │ Correction │
|
|
├─────────┼─────────────┼──────────────┤
|
|
│ │ Object: │ Correct? │
|
|
│ │ Road Crack │ YES / NO │
|
|
│ │ Confidence: │ │
|
|
│ │ 82% │ Severity: │
|
|
│ │ │ Low / Medium │
|
|
│ │ │ / High │
|
|
└─────────┴─────────────┴──────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Decision Cases
|
|
|
|
**The most important asset.**
|
|
|
|
Not images. Not videos. Not AI models.
|
|
|
|
But:
|
|
```
|
|
Observation → Evidence → Finding → Decision → Outcome → Learning
|
|
```
|
|
|
|
After a few years, hundreds of thousands of verified Decision Cases.
|
|
Not just a training dataset — a library of how real observations lead to real decisions and real outcomes.
|
|
|
|
```
|
|
Case #4232
|
|
├── Reality
|
|
├── Observation
|
|
├── Evidence
|
|
├── Finding
|
|
├── Decision
|
|
├── Outcome
|
|
└── Learning
|
|
```
|
|
|
|
All cases playable.
|
|
|
|
---
|
|
|
|
## Benchmark
|
|
|
|
Compare models:
|
|
|
|
| Model | Precision | Recall | F1 | Latency | Decision Accuracy |
|
|
|-------|-----------|--------|----|---------|-------------------|
|
|
| YOLO v8 | 0.89 | 0.87 | 0.88 | 45ms | — |
|
|
| Grounding DINO | 0.91 | 0.85 | 0.88 | 120ms | — |
|
|
| SAM | 0.88 | 0.90 | 0.89 | 200ms | — |
|
|
| Custom | 0.92 | 0.91 | 0.915 | 60ms | 0.87 |
|
|
|
|
---
|
|
|
|
## Replay
|
|
|
|
Find regressions:
|
|
|
|
```
|
|
Mission 213
|
|
├── Play
|
|
├── Show AI
|
|
├── Show Human Annotation
|
|
├── Differences
|
|
├── New Model
|
|
└── Old Model
|
|
```
|
|
|
|
---
|
|
|
|
## Validation
|
|
|
|
```
|
|
Field Trials
|
|
Scenario Tests
|
|
Decision Tests
|
|
Evidence Tests
|
|
Golden Failures
|
|
Regression Tests
|
|
```
|
|
|
|
---
|
|
|
|
## Promote Model (Not Deploy)
|
|
|
|
```
|
|
Development → Validation → Pilot → Production
|
|
```
|
|
|
|
Not "Deploy". "Promote Model".
|
|
|
|
---
|
|
|
|
## Experiments
|
|
|
|
```
|
|
Experiments
|
|
├── EP-1.0
|
|
├── DS-001
|
|
├── DS-002
|
|
├── DS-003
|
|
├── Field Trials
|
|
└── Metrics
|
|
```
|
|
|
|
Link experiment protocol to real development and validation data.
|
|
|
|
---
|
|
|
|
## Processing Pipelines
|
|
|
|
### 1. Ingestion Pipeline
|
|
|
|
**Input:** Images, video, GPS, EXIF, metadata
|
|
**Checks:** Checksums, versioning
|
|
**Output:** Raw Dataset
|
|
|
|
### 2. Dataset Pipeline
|
|
|
|
**Input:** Raw Dataset
|
|
**Checks:** Sort, deduplicate, quality control, resolution, blur detection, GPS validation
|
|
**Output:** Validated Dataset
|
|
|
|
### 3. Annotation Pipeline
|
|
|
|
**Input:** Validated Dataset
|
|
**Process:** AI suggestions, manual correction, label versions, consensus
|
|
**Output:** Verified Dataset
|
|
|
|
### 4. Training Pipeline
|
|
|
|
**Input:** Verified Dataset
|
|
**Process:** Start training, hyperparameters, checkpoints, GPU jobs
|
|
**Output:** Model Artifact
|
|
|
|
### 5. Evaluation Pipeline
|
|
|
|
**Input:** Model Artifact
|
|
**Metrics:** Precision, recall, F1, decision accuracy, regression
|
|
**Output:** Evaluation Report
|
|
|
|
### 6. Replay Pipeline
|
|
|
|
**Input:** Old missions, Model v14, Model v15
|
|
**Process:** Run both models, compare differences
|
|
**Output:** Regression Report
|
|
|
|
### 7. Decision Validation Pipeline
|
|
|
|
**Input:** Observation
|
|
**Process:** Full chain — Observation → Evidence → Finding → Decision → Business Impact
|
|
**Output:** Decision Validation Report
|
|
|
|
---
|
|
|
|
## Bucket Structure
|
|
|
|
```
|
|
raw-images/
|
|
raw-video/
|
|
missions/
|
|
datasets/
|
|
annotations/
|
|
training/
|
|
models/
|
|
replays/
|
|
evaluation/
|
|
decision-cases/
|
|
field-trials/
|
|
exports/
|
|
archives/
|
|
```
|
|
|
|
All content is versioned:
|
|
```
|
|
model-v14/
|
|
model-v15/
|
|
model-v16/
|
|
```
|
|
|
|
---
|
|
|
|
## AI Job Queue
|
|
|
|
All jobs are asynchronous:
|
|
|
|
```
|
|
Upload → Queue → Worker → GPU → Storage → Notification
|
|
```
|
|
|
|
Not synchronous API calls.
|
|
|
|
---
|
|
|
|
## What This Tool Collects
|
|
|
|
- Model training
|
|
- Annotation
|
|
- Datasets
|
|
- Replay
|
|
- Decision chains
|
|
- Validation
|
|
- Regression tests
|
|
- Experiments
|
|
- Model promotion
|
|
|
|
---
|
|
|
|
## New Developer Experience
|
|
|
|
A new AI engineer should open the repo and within minutes understand:
|
|
|
|
**"This is the tool where we build, test, and verify LandveX Control Intelligence before anything reaches production."**
|
|
|
|
---
|
|
|
|
## Relationship to Principles
|
|
|
|
- All development in Git
|
|
- All experiments reproducible
|
|
- All models traceable from training to validation to production
|
|
- Version control and traceability
|
|
|
|
---
|
|
|
|
## ändringshistoria
|
|
|
|
| Version | Datum | Beskrivning |
|
|
|---------|-------|-------------|
|
|
| 1.0 | 2026-07-02 | Initial specification for LandveX Intelligence Lab |
|
|
|
|
---
|
|
|
|
## STATUS
|
|
|
|
**SPECIFICATION — Awaiting development decision**
|