boc/quixzoom-capture-pipeline/uios/UIOS-SPECIFICATION-v1.0.md

# UIOS Specification v1.0

## Urban Intelligence Operating System

**Version:** 1.0.0  
**Date:** 2026-06-28  
**Status:** Draft  
**Author:** QUIXZOOM Engineering

---

## Table of Contents

1. [Vision & Principles](#1-vision--principles)
2. [System Architecture](#2-system-architecture)
3. [Data Model](#3-data-model)
4. [Taxonomy](#4-taxonomy)
5. [Urban Ontology](#5-urban-ontology)
6. [AI Platform](#6-ai-platform)
7. [API Contracts](#7-api-contracts)
8. [Data Collection](#8-data-collection)
9. [Security](#9-security)
10. [Operations](#10-operations)
11. [Roadmap](#11-roadmap)

---

## 1. Vision & Principles

### 1.1 Vision

UIOS is the operating system for continuous urban intelligence. It transforms raw observations into actionable knowledge about the built environment.

> "We don't just recognize objects. We understand how they relate, how they change, and how they affect each other."

### 1.2 Design Principles

| Principle | Description |
|-----------|-------------|
| **Event-Driven** | All actions are triggered by events, not polling |
| **AI-First** | AI guides every decision, from mission planning to quality control |
| **Observation-First** | Raw observations are sacred; never discard original data |
| **API-First** | Every component exposes a well-defined API |
| **Reference City** | Each city is a reference environment for specific climate/type |
| **Self-Improving** | The system learns from every observation and improves over time |

---

## 2. System Architecture

### 2.1 Six Layers

```
┌─────────────────────────────────────────┐
│  Layer 6: API                           │
│  Analytics & Customer API               │
├─────────────────────────────────────────┤
│  Layer 5: Operations                    │
│  Training Pipeline & Deployment Manager │
├─────────────────────────────────────────┤
│  Layer 4: Data                          │
│  Dataset Manager & Model Registry       │
├─────────────────────────────────────────┤
│  Layer 3: Knowledge                     │
│  UKG, OIE, Change Detection             │
├─────────────────────────────────────────┤
│  Layer 2: Intelligence                  │
│  Mission Planner, Active Learning, WS   │
├─────────────────────────────────────────┤
│  Layer 1: Data Collection               │
│  Capture Engine & Quality Control       │
└─────────────────────────────────────────┘
```

### 2.2 Component Diagram

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Zoomer    │────▶│   Capture   │────▶│   Quality   │
│   App       │     │   Engine    │     │   Control   │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
                        ┌──────────────────────┘
                        ▼
               ┌─────────────────┐
               │  Weak Supervision│
               │  Pipeline        │
               └────────┬────────┘
                        │
           ┌────────────┼────────────┐
           ▼            ▼            ▼
      ┌─────────┐ ┌─────────┐ ┌─────────┐
      │  YOLO   │ │ Ground. │ │  SAM2   │
      │  v8     │ │ DINO    │ │         │
      └─────────┘ └─────────┘ └─────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Human Review   │
               │  (if needed)    │
               └────────┬────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Object Identity │
               │  Engine (OIE)    │
               └────────┬────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Urban Knowledge │
               │  Graph (UKG)     │
               └────────┬────────┘
                        │
           ┌────────────┼────────────┐
           ▼            ▼            ▼
      ┌─────────┐ ┌─────────┐ ┌─────────┐
      │ Change  │ │Coverage │ │ Mission │
      │Detection│ │Analyzer │ │ Planner │
      └─────────┘ └─────────┘ └─────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Active Learning │
               │  Loop            │
               └─────────────────┘
```

### 2.3 Data Flows

**Observation Flow:**
```
Capture → Quality Control → Weak Supervision → Human Review →
OIE → UKG → Change Detection → Coverage Update → Mission Generation
```

**Training Flow:**
```
Gold Dataset → Training Pipeline → Model Registry →
Shadow Deployment → Evaluation → Promotion → Production
```

**Mission Flow:**
```
Coverage Gap → Mission Score → Priority Queue →
Zoomer Assignment → Capture → Quality Control
```

---

## 3. Data Model

### 3.1 Core Entities

#### Observation
```typescript
interface Observation {
  id: string;                    // Unique identifier
  objectType: string;            // Hierarchical type
  location: {
    lat: number;
    lng: number;
    accuracy: number;            // GPS accuracy in meters
  };
  timestamp: string;             // ISO 8601
  
  // Media
  media: {
    type: 'image' | 'video' | 'depth';
    url: string;
    resolution: [number, number];
    format: string;
  };
  
  // AI Analysis
  aiAnalysis: {
    detections: Detection[];
    confidence: number;
    modelVersion: string;
  };
  
  // Quality
  quality: {
    blur: number;                // 0-1
    exposure: number;            // 0-1
    noise: number;               // 0-1
    overall: number;             // 0-1
  };
  
  // Metadata
  source: {
    type: 'zoomer' | 'api' | 'import';
    zoomerId?: string;
    device?: string;
    missionId?: string;
  };
  
  // Context
  context: {
    weather: string;
    timeOfDay: 'day' | 'evening' | 'night';
    season: string;
    temperature?: number;
  };
}
```

#### Object (UKG)
```typescript
interface Object {
  id: string;                    // Unique identifier
  type: string;                  // Hierarchical type
  
  // Location
  location: {
    lat: number;
    lng: number;
    accuracy: number;
  };
  
  // Identity
  identity: {
    confidence: number;
    verificationStatus: 'unverified' | 'verified' | 'disputed';
    mergedFrom?: string[];       // Source object IDs
  };
  
  // Evidence
  evidence: string[];            // Observation IDs
  firstSeen: string;
  lastSeen: string;
  
  // Attributes
  attributes: Record<string, any>;
  
  // Relations
  relations: Relation[];
  
  // Health
  health: {
    status: 'good' | 'fair' | 'poor' | 'critical';
    score: number;               // 0-1
    lastAssessment: string;
  };
  
  // Temporal
  temporal: {
    createdAt: string;
    updatedAt: string;
    deprecatedAt?: string;
  };
}
```

#### Mission
```typescript
interface Mission {
  id: string;
  type: 'micro' | 'local' | 'regional';
  priority: 1 | 2 | 3;           // 1=critical, 2=important, 3=normal
  
  // Target
  target: {
    objectId?: string;
    category?: string;
    location?: {
      lat: number;
      lng: number;
      radius: number;
    };
  };
  
  // Instructions
  instructions: string[];
  
  // Scoring
  score: {
    total: number;
    breakdown: {
      coverageGap: number;
      informationGain: number;
      customerDemand: number;
      infrastructureCriticality: number;
      predictionUncertainty: number;
      temporalFreshness: number;
      dataQuality: number;
    };
  };
  
  // Compensation
  compensation: {
    base: number;
    final: number;
    currency: string;
  };
  
  // Status
  status: 'open' | 'assigned' | 'completed' | 'cancelled';
  assignedTo?: string;           // Zoomer ID
  
  // Metadata
  createdAt: string;
  deadline?: string;
}
```

### 3.2 Dataset Versioning

```typescript
interface DatasetVersion {
  id: string;
  name: string;
  
  // Content
  cities: string[];
  categories: string[];
  conditions: {
    timeOfDay?: string[];
    weather?: string[];
    season?: string[];
  };
  
  // Data
  observations: string[];
  annotations: string[];
  
  // Stats
  stats: {
    totalObservations: number;
    totalAnnotations: number;
    verifiedAnnotations: number;
    avgQuality: number;
  };
  
  // Lineage
  parentVersion?: string;
  modelVersion?: string;
  
  // Status
  status: 'draft' | 'committed' | 'archived';
  committedAt?: string;
  commitMessage?: string;
}
```

---

## 4. Taxonomy

### 4.1 Hierarchical Labels

```
Infrastructure
├── Road
│   ├── asphalt
│   ├── crack
│   ├── pothole
│   └── lane_marking
├── Lighting
│   ├── street_lamp
│   ├── traffic_light
│   └── flood_light
├── Utility
│   ├── electrical_cabinet
│   ├── manhole
│   ├── drain
│   └── hydrant
├── Signage
│   ├── stop
│   ├── speed
│   ├── direction
│   └── warning
├── Vegetation
│   ├── tree
│   ├── bush
│   └── grass
└── Furniture
    ├── bench
    ├── trash_can
    └── bike_rack
```

### 4.2 Attributes by Type

| Type | Attributes |
|------|-----------|
| street_lamp | height, material, paint, light_status, rust, lean |
| traffic_sign | sign_type, height, reflective, damaged |
| tree | species, height, diameter, health |
| manhole | diameter, material, condition |
| electrical_cabinet | type, condition, height |

---

## 5. Urban Ontology

### 5.1 Relations

```
street_lamp  --illuminates-->  road
road         --belongs_to-->   street_network
crosswalk    --crosses-->      road
traffic_light --regulates-->   crosswalk
electrical_cabinet --powers--> street_lamp
tree         --may_obscure-->  sign
sign         --mounted_on-->   pole
manhole      --provides_access_to--> sewer
drain        --connects_to-->  sewer
hydrant      --connected_to--> water_main
```

### 5.2 Semantic Queries

**Example 1: Impact Analysis**
```
"This street lamp is broken. Which crosswalks are affected?"

Query:
  MATCH (lamp:street_lamp {id: 'x'})-[:illuminates]->(road:road)
        <-[:crosses]-(crosswalk:crosswalk)
  RETURN crosswalk
```

**Example 2: Growth Risk**
```
"Which signs risk being obscured if trees continue growing?"

Query:
  MATCH (tree:tree)-[:may_obscure]->(sign:sign)
  WHERE tree.health = 'growing'
  RETURN tree, sign
```

**Example 3: Power Dependency**
```
"Which objects are affected if this electrical cabinet fails?"

Query:
  MATCH (cabinet:electrical_cabinet {id: 'x'})-[:powers]->(obj)
  RETURN obj
```

### 5.3 Ontology Schema

```typescript
interface Relation {
  id: string;
  type: string;                  // Relation type
  from: string;                  // Source object ID
  to: string;                    // Target object ID
  
  // Properties
  properties: {
    strength: number;            // 0-1, relation confidence
    directional: boolean;        // Is relation directional?
    temporal?: {
      validFrom: string;
      validTo?: string;
    };
  };
  
  // Discovery
  discoveredBy: string;          // Model or human
  discoveredAt: string;
  verified: boolean;
}
```

---

## 6. AI Platform

### 6.1 Weak Supervision

| Model | Weight | Status | Classes |
|-------|--------|--------|---------|
| YOLOv8 | 30% | Available | COCO |
| Grounding DINO | 30% | Planned | Text-prompted |
| SAM2 | 20% | Planned | Segmentation |
| OCR | 10% | Planned | Text |
| Depth | 10% | Planned | 3D position |

### 6.2 Human-in-the-Loop

```
AI Proposal → Confidence Check → Human Review → Gold Dataset
                │
                └─> High confidence (>0.9) → Auto-accept
                └─> Medium confidence (0.7-0.9) → Suggest
                └─> Low confidence (<0.7) → Require review
```

### 6.3 Active Learning Loop

```
Mission Planner → Zoomer Assignment → Capture →
Quality Control → Weak Supervision → Human Review →
Urban Knowledge Graph → Coverage Analyzer →
Model Evaluation → Knowledge Gap Detection →
Mission Planner
```

### 6.4 Model Registry

| Field | Description |
|-------|-------------|
| modelId | Unique identifier |
| name | Model name |
| version | Semantic version |
| datasetVersion | Training data version |
| metrics | Precision, Recall, mAP |
| latency | Inference time (ms) |
| size | Model size (MB) |
| status | registered/shadow/production |

### 6.5 Shadow Deployment

1. Train new model
2. Register in Model Registry
3. Deploy in shadow mode (10% traffic)
4. Evaluate against current model
5. If improvement > 5%: promote to production
6. If degradation: rollback

---

## 7. API Contracts

### 7.1 REST API

#### Observations
```
POST /api/v1/observations
  Body: Observation
  Response: { id, status, quality }

GET /api/v1/observations/{id}
  Response: Observation

GET /api/v1/observations
  Query: cityId, type, bbox, timeRange
  Response: Observation[]
```

#### Objects
```
GET /api/v1/objects/{id}
  Response: Object

GET /api/v1/objects
  Query: cityId, type, bbox
  Response: Object[]

GET /api/v1/objects/{id}/relations
  Response: Relation[]
```

#### Missions
```
POST /api/v1/missions
  Body: MissionRequest
  Response: Mission[]

GET /api/v1/missions/{id}
  Response: Mission

POST /api/v1/missions/{id}/complete
  Body: MissionResult
  Response: { status, compensation }
```

#### Analytics
```
GET /api/v1/analytics/coverage/{cityId}
  Response: CoverageReport

GET /api/v1/analytics/quality
  Query: cityId, timeRange
  Response: QualityReport

GET /api/v1/analytics/models
  Response: Model[]
```

### 7.2 WebSocket API

```
CONNECT /ws/v1

// Client → Server
{
  "type": "subscribe",
  "channel": "missions",
  "cityId": "bangkok"
}

// Server → Client
{
  "type": "mission:created",
  "data": Mission
}

{
  "type": "observation:processed",
  "data": {
    "observationId": "...",
    "objectId": "...",
    "action": "merge|new"
  }
}
```

### 7.3 Event Schema

```typescript
interface UIOSEvent {
  id: string;
  type: string;
  timestamp: string;
  source: string;
  
  data: Record<string, any>;
  
  // Tracing
  traceId: string;
  parentId?: string;
  
  // Context
  cityId?: string;
  missionId?: string;
  observationId?: string;
}
```

---

## 8. Data Collection

### 8.1 Capture Engine

**Supported Inputs:**
- Photo (JPEG, HEIC)
- Video (MP4, MOV)
- Depth map (LiDAR, ToF)
- GPS track (GPX)

**Quality Requirements:**
- Resolution: minimum 1080p
- GPS accuracy: < 5 meters
- Timestamp: UTC with timezone
- Metadata: device, settings, conditions

### 8.2 Mission Types

| Level | Duration | Example |
|-------|----------|---------|
| Micro | Seconds | "Rotate camera 30° right" |
| Local | Minutes | "Document all street lamps on this street" |
| Regional | Days | "Night inventory of Bangkok" |

### 8.3 Quality Control

**Automated Checks:**
- GPS accuracy
- Image blur
- Exposure
- Duplicate detection
- Temporal consistency

**Human Review Triggers:**
- Low confidence (< 0.7)
- Novel object type
- Change detected
- Customer request

---

## 9. Security

### 9.1 Authentication

- JWT tokens for API access
- API keys for device authentication
- OAuth 2.0 for customer access

### 9.2 Authorization

| Role | Permissions |
|------|-------------|
| Zoomer | Submit observations, view missions |
| Reviewer | Review annotations, verify objects |
| Admin | Manage cities, models, users |
| Customer | Create requests, view analytics |

### 9.3 Encryption

- Data at rest: AES-256
- Data in transit: TLS 1.3
- Sensitive fields: Field-level encryption

### 9.4 Audit Logging

All actions logged:
- User ID
- Timestamp
- Action type
- Resource affected
- Before/after state

---

## 10. Operations

### 10.1 Deployment

```
Development → Staging → Production
     │            │           │
     └─ Unit tests ┘           │
          └─ Integration tests ┘
               └─ Shadow deployment
                    └─ Full rollout
```

### 10.2 Scaling

| Component | Scaling Strategy |
|-----------|-----------------|
| API | Horizontal (load balancer) |
| OIE | Horizontal (sharding by location) |
| UKG | Vertical (memory-optimized) |
| Training | GPU cluster (Kubernetes) |

### 10.3 Backup

- Daily snapshots of UKG
- Weekly full backups
- Point-in-time recovery
- Cross-region replication

### 10.4 Monitoring

| Metric | Alert Threshold |
|--------|----------------|
| API latency | P99 > 500ms |
| Error rate | > 1% |
| Queue depth | > 1000 items |
| Model drift | > 5% |

### 10.5 Replay

All events stored for replay:
- Debugging
- Benchmarking
- Model comparison
- Audit

### 10.6 Benchmark

Continuous benchmarking:
- Precision / Recall
- mAP@50 / mAP@75
- Latency (P50, P99)
- Throughput

---

## 11. Roadmap

### v1.0 (Current)
- ✅ Core UIOS architecture
- ✅ Object Identity Engine
- ✅ Urban Knowledge Graph
- ✅ Mission Planner
- ✅ Weak Supervision
- ✅ Dataset Manager

### v1.5 (Q3 2026)
- Grounding DINO integration
- SAM2 segmentation
- Urban Ontology (relations)
- Customer API
- Mobile app v2

### v2.0 (Q4 2026)
- Multi-city deployment
- Real-time collaboration
- Advanced analytics
- Model marketplace
- Shadow deployment automation

### v3.0 (2027)
- Autonomous mission planning
- Predictive maintenance
- City digital twin
- Cross-city learning
- Full Urban Ontology

---

## Appendix A: Reference Cities

| City | Country | Type | Climate |
|------|---------|------|---------|
| Bangkok | TH | Tropical megacity | Tropical |
| Torrevieja | ES | Mediterranean coastal | Mediterranean |
| Stockholm | SE | Nordic | Temperate |
| Tokyo | JP | Megacity | Temperate |
| Dubai | AE | Modern desert | Desert |

## Appendix B: Glossary

| Term | Definition |
|------|-----------|
| UIOS | Urban Intelligence Operating System |
| UKG | Urban Knowledge Graph |
| OIE | Object Identity Engine |
| WS | Weak Supervision |
| Reference City | A city used as reference for a specific climate/type |
| Coverage Score | Percentage of expected objects that have been observed |
| Data Value Score | Combined score indicating the value of an observation |
| Mission Score | Combined score indicating the priority of a mission |
| Shadow Deployment | Running new model in parallel with production model |

---

**End of Specification**