boc/quixzoom-capture-pipeline/uios/UIOS-SPECIFICATION-v1.0.md

UIOS Specification v1.0

Urban Intelligence Operating System

Version: 1.0.0
Date: 2026-06-28
Status: Draft
Author: QUIXZOOM Engineering

---

Table of Contents

1. Vision & Principles
2. System Architecture
3. Data Model
4. Taxonomy
5. Urban Ontology
6. AI Platform
7. API Contracts
8. Data Collection
9. Security
10. Operations
11. Roadmap

---

1. Vision & Principles

1.1 Vision

UIOS is the operating system for continuous urban intelligence. It transforms raw observations into actionable knowledge about the built environment.

"We don't just recognize objects. We understand how they relate, how they change, and how they affect each other."

1.2 Design Principles

Principle	Description
Event-Driven	All actions are triggered by events, not polling
AI-First	AI guides every decision, from mission planning to quality control
Observation-First	Raw observations are sacred; never discard original data
API-First	Every component exposes a well-defined API
Reference City	Each city is a reference environment for specific climate/type
Self-Improving	The system learns from every observation and improves over time

---

2. System Architecture

2.1 Six Layers

┌─────────────────────────────────────────┐
│  Layer 6: API                           │
│  Analytics & Customer API               │
├─────────────────────────────────────────┤
│  Layer 5: Operations                    │
│  Training Pipeline & Deployment Manager │
├─────────────────────────────────────────┤
│  Layer 4: Data                          │
│  Dataset Manager & Model Registry       │
├─────────────────────────────────────────┤
│  Layer 3: Knowledge                     │
│  UKG, OIE, Change Detection             │
├─────────────────────────────────────────┤
│  Layer 2: Intelligence                  │
│  Mission Planner, Active Learning, WS   │
├─────────────────────────────────────────┤
│  Layer 1: Data Collection               │
│  Capture Engine & Quality Control       │
└─────────────────────────────────────────┘

2.2 Component Diagram

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Zoomer    │────▶│   Capture   │────▶│   Quality   │
│   App       │     │   Engine    │     │   Control   │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
                        ┌──────────────────────┘
                        ▼
               ┌─────────────────┐
               │  Weak Supervision│
               │  Pipeline        │
               └────────┬────────┘
                        │
           ┌────────────┼────────────┐
           ▼            ▼            ▼
      ┌─────────┐ ┌─────────┐ ┌─────────┐
      │  YOLO   │ │ Ground. │ │  SAM2   │
      │  v8     │ │ DINO    │ │         │
      └─────────┘ └─────────┘ └─────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Human Review   │
               │  (if needed)    │
               └────────┬────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Object Identity │
               │  Engine (OIE)    │
               └────────┬────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Urban Knowledge │
               │  Graph (UKG)     │
               └────────┬────────┘
                        │
           ┌────────────┼────────────┐
           ▼            ▼            ▼
      ┌─────────┐ ┌─────────┐ ┌─────────┐
      │ Change  │ │Coverage │ │ Mission │
      │Detection│ │Analyzer │ │ Planner │
      └─────────┘ └─────────┘ └─────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Active Learning │
               │  Loop            │
               └─────────────────┘

2.3 Data Flows

Observation Flow:

Capture → Quality Control → Weak Supervision → Human Review →
OIE → UKG → Change Detection → Coverage Update → Mission Generation

Training Flow:

Gold Dataset → Training Pipeline → Model Registry →
Shadow Deployment → Evaluation → Promotion → Production

Mission Flow:

Coverage Gap → Mission Score → Priority Queue →
Zoomer Assignment → Capture → Quality Control

---

3. Data Model

3.1 Core Entities

Observation

interface Observation {
  id: string;                    // Unique identifier
  objectType: string;            // Hierarchical type
  location: {
    lat: number;
    lng: number;
    accuracy: number;            // GPS accuracy in meters
  };
  timestamp: string;             // ISO 8601
  
  // Media
  media: {
    type: 'image' | 'video' | 'depth';
    url: string;
    resolution: [number, number];
    format: string;
  };
  
  // AI Analysis
  aiAnalysis: {
    detections: Detection[];
    confidence: number;
    modelVersion: string;
  };
  
  // Quality
  quality: {
    blur: number;                // 0-1
    exposure: number;            // 0-1
    noise: number;               // 0-1
    overall: number;             // 0-1
  };
  
  // Metadata
  source: {
    type: 'zoomer' | 'api' | 'import';
    zoomerId?: string;
    device?: string;
    missionId?: string;
  };
  
  // Context
  context: {
    weather: string;
    timeOfDay: 'day' | 'evening' | 'night';
    season: string;
    temperature?: number;
  };
}

Object (UKG)

interface Object {
  id: string;                    // Unique identifier
  type: string;                  // Hierarchical type
  
  // Location
  location: {
    lat: number;
    lng: number;
    accuracy: number;
  };
  
  // Identity
  identity: {
    confidence: number;
    verificationStatus: 'unverified' | 'verified' | 'disputed';
    mergedFrom?: string[];       // Source object IDs
  };
  
  // Evidence
  evidence: string[];            // Observation IDs
  firstSeen: string;
  lastSeen: string;
  
  // Attributes
  attributes: Record<string, any>;
  
  // Relations
  relations: Relation[];
  
  // Health
  health: {
    status: 'good' | 'fair' | 'poor' | 'critical';
    score: number;               // 0-1
    lastAssessment: string;
  };
  
  // Temporal
  temporal: {
    createdAt: string;
    updatedAt: string;
    deprecatedAt?: string;
  };
}

Mission

interface Mission {
  id: string;
  type: 'micro' | 'local' | 'regional';
  priority: 1 | 2 | 3;           // 1=critical, 2=important, 3=normal
  
  // Target
  target: {
    objectId?: string;
    category?: string;
    location?: {
      lat: number;
      lng: number;
      radius: number;
    };
  };
  
  // Instructions
  instructions: string[];
  
  // Scoring
  score: {
    total: number;
    breakdown: {
      coverageGap: number;
      informationGain: number;
      customerDemand: number;
      infrastructureCriticality: number;
      predictionUncertainty: number;
      temporalFreshness: number;
      dataQuality: number;
    };
  };
  
  // Compensation
  compensation: {
    base: number;
    final: number;
    currency: string;
  };
  
  // Status
  status: 'open' | 'assigned' | 'completed' | 'cancelled';
  assignedTo?: string;           // Zoomer ID
  
  // Metadata
  createdAt: string;
  deadline?: string;
}

3.2 Dataset Versioning

interface DatasetVersion {
  id: string;
  name: string;
  
  // Content
  cities: string[];
  categories: string[];
  conditions: {
    timeOfDay?: string[];
    weather?: string[];
    season?: string[];
  };
  
  // Data
  observations: string[];
  annotations: string[];
  
  // Stats
  stats: {
    totalObservations: number;
    totalAnnotations: number;
    verifiedAnnotations: number;
    avgQuality: number;
  };
  
  // Lineage
  parentVersion?: string;
  modelVersion?: string;
  
  // Status
  status: 'draft' | 'committed' | 'archived';
  committedAt?: string;
  commitMessage?: string;
}

---

4. Taxonomy

4.1 Hierarchical Labels

Infrastructure
├── Road
│   ├── asphalt
│   ├── crack
│   ├── pothole
│   └── lane_marking
├── Lighting
│   ├── street_lamp
│   ├── traffic_light
│   └── flood_light
├── Utility
│   ├── electrical_cabinet
│   ├── manhole
│   ├── drain
│   └── hydrant
├── Signage
│   ├── stop
│   ├── speed
│   ├── direction
│   └── warning
├── Vegetation
│   ├── tree
│   ├── bush
│   └── grass
└── Furniture
    ├── bench
    ├── trash_can
    └── bike_rack

4.2 Attributes by Type

Type	Attributes
street_lamp	height, material, paint, light_status, rust, lean
traffic_sign	sign_type, height, reflective, damaged
tree	species, height, diameter, health
manhole	diameter, material, condition
electrical_cabinet	type, condition, height

---

5. Urban Ontology

5.1 Relations

street_lamp  --illuminates-->  road
road         --belongs_to-->   street_network
crosswalk    --crosses-->      road
traffic_light --regulates-->   crosswalk
electrical_cabinet --powers--> street_lamp
tree         --may_obscure-->  sign
sign         --mounted_on-->   pole
manhole      --provides_access_to--> sewer
drain        --connects_to-->  sewer
hydrant      --connected_to--> water_main

5.2 Semantic Queries

Example 1: Impact Analysis

"This street lamp is broken. Which crosswalks are affected?"

Query:
  MATCH (lamp:street_lamp {id: 'x'})-[:illuminates]->(road:road)
        <-[:crosses]-(crosswalk:crosswalk)
  RETURN crosswalk

Example 2: Growth Risk

"Which signs risk being obscured if trees continue growing?"

Query:
  MATCH (tree:tree)-[:may_obscure]->(sign:sign)
  WHERE tree.health = 'growing'
  RETURN tree, sign

Example 3: Power Dependency

"Which objects are affected if this electrical cabinet fails?"

Query:
  MATCH (cabinet:electrical_cabinet {id: 'x'})-[:powers]->(obj)
  RETURN obj

5.3 Ontology Schema

interface Relation {
  id: string;
  type: string;                  // Relation type
  from: string;                  // Source object ID
  to: string;                    // Target object ID
  
  // Properties
  properties: {
    strength: number;            // 0-1, relation confidence
    directional: boolean;        // Is relation directional?
    temporal?: {
      validFrom: string;
      validTo?: string;
    };
  };
  
  // Discovery
  discoveredBy: string;          // Model or human
  discoveredAt: string;
  verified: boolean;
}

---

6. AI Platform

6.1 Weak Supervision

Model	Weight	Status	Classes
YOLOv8	30%	Available	COCO
Grounding DINO	30%	Planned	Text-prompted
SAM2	20%	Planned	Segmentation
OCR	10%	Planned	Text
Depth	10%	Planned	3D position

6.2 Human-in-the-Loop

AI Proposal → Confidence Check → Human Review → Gold Dataset
                │
                └─> High confidence (>0.9) → Auto-accept
                └─> Medium confidence (0.7-0.9) → Suggest
                └─> Low confidence (<0.7) → Require review

6.3 Active Learning Loop

Mission Planner → Zoomer Assignment → Capture →
Quality Control → Weak Supervision → Human Review →
Urban Knowledge Graph → Coverage Analyzer →
Model Evaluation → Knowledge Gap Detection →
Mission Planner

6.4 Model Registry

Field	Description
modelId	Unique identifier
name	Model name
version	Semantic version
datasetVersion	Training data version
metrics	Precision, Recall, mAP
latency	Inference time (ms)
size	Model size (MB)
status	registered/shadow/production

6.5 Shadow Deployment

1. Train new model
2. Register in Model Registry
3. Deploy in shadow mode (10% traffic)
4. Evaluate against current model
5. If improvement > 5%: promote to production
6. If degradation: rollback

---

7. API Contracts

7.1 REST API

Observations

POST /api/v1/observations
  Body: Observation
  Response: { id, status, quality }

GET /api/v1/observations/{id}
  Response: Observation

GET /api/v1/observations
  Query: cityId, type, bbox, timeRange
  Response: Observation[]

Objects

GET /api/v1/objects/{id}
  Response: Object

GET /api/v1/objects
  Query: cityId, type, bbox
  Response: Object[]

GET /api/v1/objects/{id}/relations
  Response: Relation[]

Missions

POST /api/v1/missions
  Body: MissionRequest
  Response: Mission[]

GET /api/v1/missions/{id}
  Response: Mission

POST /api/v1/missions/{id}/complete
  Body: MissionResult
  Response: { status, compensation }

Analytics

GET /api/v1/analytics/coverage/{cityId}
  Response: CoverageReport

GET /api/v1/analytics/quality
  Query: cityId, timeRange
  Response: QualityReport

GET /api/v1/analytics/models
  Response: Model[]

7.2 WebSocket API

CONNECT /ws/v1

// Client → Server
{
  "type": "subscribe",
  "channel": "missions",
  "cityId": "bangkok"
}

// Server → Client
{
  "type": "mission:created",
  "data": Mission
}

{
  "type": "observation:processed",
  "data": {
    "observationId": "...",
    "objectId": "...",
    "action": "merge|new"
  }
}

7.3 Event Schema

interface UIOSEvent {
  id: string;
  type: string;
  timestamp: string;
  source: string;
  
  data: Record<string, any>;
  
  // Tracing
  traceId: string;
  parentId?: string;
  
  // Context
  cityId?: string;
  missionId?: string;
  observationId?: string;
}

---

8. Data Collection

8.1 Capture Engine

Supported Inputs:

• Photo (JPEG, HEIC)
• Video (MP4, MOV)
• Depth map (LiDAR, ToF)
• GPS track (GPX)

Quality Requirements:

• Resolution: minimum 1080p
• GPS accuracy: < 5 meters
• Timestamp: UTC with timezone
• Metadata: device, settings, conditions

8.2 Mission Types

Level	Duration	Example
Micro	Seconds	"Rotate camera 30° right"
Local	Minutes	"Document all street lamps on this street"
Regional	Days	"Night inventory of Bangkok"

8.3 Quality Control

Automated Checks:

• GPS accuracy
• Image blur
• Exposure
• Duplicate detection
• Temporal consistency

Human Review Triggers:

• Low confidence (< 0.7)
• Novel object type
• Change detected
• Customer request

---

9. Security

9.1 Authentication

• JWT tokens for API access
• API keys for device authentication
• OAuth 2.0 for customer access

9.2 Authorization

Role	Permissions
Zoomer	Submit observations, view missions
Reviewer	Review annotations, verify objects
Admin	Manage cities, models, users
Customer	Create requests, view analytics

9.3 Encryption

• Data at rest: AES-256
• Data in transit: TLS 1.3
• Sensitive fields: Field-level encryption

9.4 Audit Logging

All actions logged:

• User ID
• Timestamp
• Action type
• Resource affected
• Before/after state

---

10. Operations

10.1 Deployment

Development → Staging → Production
     │            │           │
     └─ Unit tests ┘           │
          └─ Integration tests ┘
               └─ Shadow deployment
                    └─ Full rollout

10.2 Scaling

Component	Scaling Strategy
API	Horizontal (load balancer)
OIE	Horizontal (sharding by location)
UKG	Vertical (memory-optimized)
Training	GPU cluster (Kubernetes)

10.3 Backup

• Daily snapshots of UKG
• Weekly full backups
• Point-in-time recovery
• Cross-region replication

10.4 Monitoring

Metric	Alert Threshold
API latency	P99 > 500ms
Error rate	> 1%
Queue depth	> 1000 items
Model drift	> 5%

10.5 Replay

All events stored for replay:

• Debugging
• Benchmarking
• Model comparison
• Audit

10.6 Benchmark

Continuous benchmarking:

• Precision / Recall
• mAP@50 / mAP@75
• Latency (P50, P99)
• Throughput

---

11. Roadmap

v1.0 (Current)

• ✅ Core UIOS architecture
• ✅ Object Identity Engine
• ✅ Urban Knowledge Graph
• ✅ Mission Planner
• ✅ Weak Supervision
• ✅ Dataset Manager

v1.5 (Q3 2026)

• Grounding DINO integration
• SAM2 segmentation
• Urban Ontology (relations)
• Customer API
• Mobile app v2

v2.0 (Q4 2026)

• Multi-city deployment
• Real-time collaboration
• Advanced analytics
• Model marketplace
• Shadow deployment automation

v3.0 (2027)

• Autonomous mission planning
• Predictive maintenance
• City digital twin
• Cross-city learning
• Full Urban Ontology

---

Appendix A: Reference Cities

City	Country	Type	Climate
Bangkok	TH	Tropical megacity	Tropical
Torrevieja	ES	Mediterranean coastal	Mediterranean
Stockholm	SE	Nordic	Temperate
Tokyo	JP	Megacity	Temperate
Dubai	AE	Modern desert	Desert

Appendix B: Glossary

Term	Definition
UIOS	Urban Intelligence Operating System
UKG	Urban Knowledge Graph
OIE	Object Identity Engine
WS	Weak Supervision
Reference City	A city used as reference for a specific climate/type
Coverage Score	Percentage of expected objects that have been observed
Data Value Score	Combined score indicating the value of an observation
Mission Score	Combined score indicating the priority of a mission
Shadow Deployment	Running new model in parallel with production model

---

End of Specification