From 9a817ac82a6bf7859f513988687645b5252ffc26 Mon Sep 17 00:00:00 2001 From: Bernt Date: Thu, 2 Jul 2026 12:26:00 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20LandveX=20Intelligence=20Lab=20v1.1=20?= =?UTF-8?q?=E2=80=94=20Architecture=20+=20Pipelines=20+=20Immutability?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Added architecture diagram: - Developer UI / Training API / Experiment API - Event Bus / Queue - Storage Layer (S3/R2 + PostgreSQL + Neo4j) - Processing Pipelines - Seven separate pipelines (not one big AI loop): 1. Ingestion — raw images/video/GPS/EXIF → Raw Dataset 2. Dataset — sort, deduplicate, quality control → Validated Dataset 3. Annotation — AI suggestions, manual correction → Verified Dataset 4. Training — hyperparameters, checkpoints, GPU → Model Artifact 5. Evaluation — precision, recall, F1, decision accuracy → Report 6. Replay — compare model versions → Regression Report 7. Decision Validation — full chain → Validation Report - Bucket structure: raw-images, raw-video, missions, datasets, annotations, training, models, replays, evaluation, decision-cases, field-trials, exports, archives - AI Job Queue: all jobs asynchronous (Upload → Queue → Worker → GPU → Storage → Notification) - Core principles updated: - All artifacts immutable and versioned - Every change traceable to experiment, model, dataset, decision - Dashboard shows: Datasets (Healthy/Needs Review/Corrupted), Training Jobs, Decision Cases, Replay Jobs - Decision Cases emphasized as most important asset Rationale: Separate pipelines make system easier to debug, improve, and swap components. Immutability aligns with E-001 Git/traceability principles. Decision Cases become the unique asset over time. --- docs/design/LANDVEX_INTELLIGENCE_LAB.md | 155 +++++++++++++++++++++--- 1 file changed, 141 insertions(+), 14 deletions(-) diff --git a/docs/design/LANDVEX_INTELLIGENCE_LAB.md b/docs/design/LANDVEX_INTELLIGENCE_LAB.md index 6d48b7385..1368a988e 100644 --- a/docs/design/LANDVEX_INTELLIGENCE_LAB.md +++ b/docs/design/LANDVEX_INTELLIGENCE_LAB.md @@ -10,12 +10,22 @@ --- -## Core Principle +## Core Principles **LandveX Intelligence Lab does not produce AI models. It produces verified Control Intelligence.** This is an internal tool. Never a customer product. +**All artifacts are immutable and versioned.** +- Raw images are never modified +- Annotations are versioned +- Models are versioned +- Evaluation reports are versioned +- Decision Cases are versioned +- Replay results are saved as new artifacts + +**Every change must be traceable to a specific experiment, model version, dataset, and decision.** + --- ## Repository @@ -31,6 +41,28 @@ Separate from: --- +## Architecture + +``` +LandveX Intelligence Lab +│ +┌─────────────────┼─────────────────┐ +│ │ │ +Developer UI Training API Experiment API +│ │ │ +└─────────────────┴─────────────────┘ +│ +Event Bus / Queue +│ +───────────────────────────────────────────────────── +Storage Layer +Images │ Videos │ Missions │ Models │ Logs +S3/R2 Buckets + PostgreSQL + Neo4j +───────────────────────────────────────────────────── +│ +Processing Pipelines +``` + ## Navigation ``` @@ -43,7 +75,7 @@ Dashboard ├── Decision Cases ├── Replay ├── Validation -├── Deploy (Promote Model) +├── Promote Model └── Settings ``` @@ -55,26 +87,26 @@ Shows AI status, not business data. ``` ┌─────────────────────────────────────┐ -│ Models │ -│ Detection v12 │ -│ Segmentation v5 │ -│ OCR v3 │ -│ Decision Model v1.0 │ -├─────────────────────────────────────┤ │ Datasets │ -│ Roads, Buildings, Signs │ -│ Vegetation, Drainage │ +│ Healthy: 12 │ +│ Needs Review: 3 │ +│ Corrupted: 0 │ ├─────────────────────────────────────┤ │ Training Jobs │ │ Running: 2 │ │ Queued: 1 │ -│ Failed: 0 │ │ Completed: 47 │ +│ Failed: 0 │ ├─────────────────────────────────────┤ │ Decision Cases │ │ Validated: 23 │ │ Pending: 5 │ │ Rejected: 2 │ +├─────────────────────────────────────┤ +│ Replay Jobs │ +│ Ready: 8 │ +│ Running: 1 │ +│ Finished: 34 │ └─────────────────────────────────────┘ ``` @@ -108,9 +140,19 @@ Video → Frame → Bounding boxes → Detected objects → Evidence → Finding --- -## Decision Case +## Decision Cases -First-class objects: +**The most important asset.** + +Not images. Not videos. Not AI models. + +But: +``` +Observation → Evidence → Finding → Decision → Outcome → Learning +``` + +After a few years, hundreds of thousands of verified Decision Cases. +Not just a training dataset — a library of how real observations lead to real decisions and real outcomes. ``` Case #4232 @@ -169,7 +211,7 @@ Regression Tests --- -## Deploy (Promote Model) +## Promote Model (Not Deploy) ``` Development → Validation → Pilot → Production @@ -195,6 +237,91 @@ Link experiment protocol to real development and validation data. --- +## Processing Pipelines + +### 1. Ingestion Pipeline + +**Input:** Images, video, GPS, EXIF, metadata +**Checks:** Checksums, versioning +**Output:** Raw Dataset + +### 2. Dataset Pipeline + +**Input:** Raw Dataset +**Checks:** Sort, deduplicate, quality control, resolution, blur detection, GPS validation +**Output:** Validated Dataset + +### 3. Annotation Pipeline + +**Input:** Validated Dataset +**Process:** AI suggestions, manual correction, label versions, consensus +**Output:** Verified Dataset + +### 4. Training Pipeline + +**Input:** Verified Dataset +**Process:** Start training, hyperparameters, checkpoints, GPU jobs +**Output:** Model Artifact + +### 5. Evaluation Pipeline + +**Input:** Model Artifact +**Metrics:** Precision, recall, F1, decision accuracy, regression +**Output:** Evaluation Report + +### 6. Replay Pipeline + +**Input:** Old missions, Model v14, Model v15 +**Process:** Run both models, compare differences +**Output:** Regression Report + +### 7. Decision Validation Pipeline + +**Input:** Observation +**Process:** Full chain — Observation → Evidence → Finding → Decision → Business Impact +**Output:** Decision Validation Report + +--- + +## Bucket Structure + +``` +raw-images/ +raw-video/ +missions/ +datasets/ +annotations/ +training/ +models/ +replays/ +evaluation/ +decision-cases/ +field-trials/ +exports/ +archives/ +``` + +All content is versioned: +``` +model-v14/ +model-v15/ +model-v16/ +``` + +--- + +## AI Job Queue + +All jobs are asynchronous: + +``` +Upload → Queue → Worker → GPU → Storage → Notification +``` + +Not synchronous API calls. + +--- + ## What This Tool Collects - Model training