diff --git a/docs/design/LANDVEX_INTELLIGENCE_LAB.md b/docs/design/LANDVEX_INTELLIGENCE_LAB.md index 6d48b7385..1368a988e 100644 --- a/docs/design/LANDVEX_INTELLIGENCE_LAB.md +++ b/docs/design/LANDVEX_INTELLIGENCE_LAB.md @@ -10,12 +10,22 @@ --- -## Core Principle +## Core Principles **LandveX Intelligence Lab does not produce AI models. It produces verified Control Intelligence.** This is an internal tool. Never a customer product. +**All artifacts are immutable and versioned.** +- Raw images are never modified +- Annotations are versioned +- Models are versioned +- Evaluation reports are versioned +- Decision Cases are versioned +- Replay results are saved as new artifacts + +**Every change must be traceable to a specific experiment, model version, dataset, and decision.** + --- ## Repository @@ -31,6 +41,28 @@ Separate from: --- +## Architecture + +``` +LandveX Intelligence Lab +│ +┌─────────────────┼─────────────────┐ +│ │ │ +Developer UI Training API Experiment API +│ │ │ +└─────────────────┴─────────────────┘ +│ +Event Bus / Queue +│ +───────────────────────────────────────────────────── +Storage Layer +Images │ Videos │ Missions │ Models │ Logs +S3/R2 Buckets + PostgreSQL + Neo4j +───────────────────────────────────────────────────── +│ +Processing Pipelines +``` + ## Navigation ``` @@ -43,7 +75,7 @@ Dashboard ├── Decision Cases ├── Replay ├── Validation -├── Deploy (Promote Model) +├── Promote Model └── Settings ``` @@ -55,26 +87,26 @@ Shows AI status, not business data. ``` ┌─────────────────────────────────────┐ -│ Models │ -│ Detection v12 │ -│ Segmentation v5 │ -│ OCR v3 │ -│ Decision Model v1.0 │ -├─────────────────────────────────────┤ │ Datasets │ -│ Roads, Buildings, Signs │ -│ Vegetation, Drainage │ +│ Healthy: 12 │ +│ Needs Review: 3 │ +│ Corrupted: 0 │ ├─────────────────────────────────────┤ │ Training Jobs │ │ Running: 2 │ │ Queued: 1 │ -│ Failed: 0 │ │ Completed: 47 │ +│ Failed: 0 │ ├─────────────────────────────────────┤ │ Decision Cases │ │ Validated: 23 │ │ Pending: 5 │ │ Rejected: 2 │ +├─────────────────────────────────────┤ +│ Replay Jobs │ +│ Ready: 8 │ +│ Running: 1 │ +│ Finished: 34 │ └─────────────────────────────────────┘ ``` @@ -108,9 +140,19 @@ Video → Frame → Bounding boxes → Detected objects → Evidence → Finding --- -## Decision Case +## Decision Cases -First-class objects: +**The most important asset.** + +Not images. Not videos. Not AI models. + +But: +``` +Observation → Evidence → Finding → Decision → Outcome → Learning +``` + +After a few years, hundreds of thousands of verified Decision Cases. +Not just a training dataset — a library of how real observations lead to real decisions and real outcomes. ``` Case #4232 @@ -169,7 +211,7 @@ Regression Tests --- -## Deploy (Promote Model) +## Promote Model (Not Deploy) ``` Development → Validation → Pilot → Production @@ -195,6 +237,91 @@ Link experiment protocol to real development and validation data. --- +## Processing Pipelines + +### 1. Ingestion Pipeline + +**Input:** Images, video, GPS, EXIF, metadata +**Checks:** Checksums, versioning +**Output:** Raw Dataset + +### 2. Dataset Pipeline + +**Input:** Raw Dataset +**Checks:** Sort, deduplicate, quality control, resolution, blur detection, GPS validation +**Output:** Validated Dataset + +### 3. Annotation Pipeline + +**Input:** Validated Dataset +**Process:** AI suggestions, manual correction, label versions, consensus +**Output:** Verified Dataset + +### 4. Training Pipeline + +**Input:** Verified Dataset +**Process:** Start training, hyperparameters, checkpoints, GPU jobs +**Output:** Model Artifact + +### 5. Evaluation Pipeline + +**Input:** Model Artifact +**Metrics:** Precision, recall, F1, decision accuracy, regression +**Output:** Evaluation Report + +### 6. Replay Pipeline + +**Input:** Old missions, Model v14, Model v15 +**Process:** Run both models, compare differences +**Output:** Regression Report + +### 7. Decision Validation Pipeline + +**Input:** Observation +**Process:** Full chain — Observation → Evidence → Finding → Decision → Business Impact +**Output:** Decision Validation Report + +--- + +## Bucket Structure + +``` +raw-images/ +raw-video/ +missions/ +datasets/ +annotations/ +training/ +models/ +replays/ +evaluation/ +decision-cases/ +field-trials/ +exports/ +archives/ +``` + +All content is versioned: +``` +model-v14/ +model-v15/ +model-v16/ +``` + +--- + +## AI Job Queue + +All jobs are asynchronous: + +``` +Upload → Queue → Worker → GPU → Storage → Notification +``` + +Not synchronous API calls. + +--- + ## What This Tool Collects - Model training