Unified Device Response Protocol - Architecture Overview¶

Date: 2025-12-08 Version: 1.0 Status: Design Complete, Ready for Implementation Project Owner: Shota Takahashi

Project Vision¶

Final State (v1.12.0+):

┌─────────────────────────────────────────┐
│     OSECHI Hardware (ESP32)             │
│  ┌────────────────────────────────────┐ │
│  │  Response / Event Processing       │ │
│  │  ├─ text_command_handlers          │ │
│  │  ├─ cosmic_detector.cpp            │ │
│  │  └─ stream_formatter.cpp           │ │
│  └────────────────────────────────────┘ │
│                  │                      │
│                  ▼                      │
│  ┌────────────────────────────────────┐ │
│  │  Unified Schema Layer              │ │
│  │  └─ device_response_t              │ │
│  │     (JsonDocument payload)         │ │
│  └────────────────────────────────────┘ │
│                  │                      │
│                  ▼ JSONL                │
└──────────────── Serial ────────────────┘
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
    kazunoko    JavaScript    Rust
     (Python)   Client        Client
        │           │           │
        └───────────┼───────────┘
                    ▼
        Unified Parser (Schema-based)
                    │
                    ▼
           Application Logic

Key Achievement: Single schema definition → All clients can implement unified parser

Core Design Principles¶

1. Single Schema for All Output¶

Every message from OSECHI follows:

{
  "type": "response" | "event",
  "status": "ok" | "error",
  "sent_at": <unix_timestamp>,
  "<payload>": <data>
}

Benefit: Client code can be language-agnostic, implementation-agnostic.

2. No Redundant Wrapper Classes¶

❌ Avoided: Payload wrapper class (would add unnecessary indirection)
✅ Used: JsonDocument directly (already provides flexibility)

Rationale: YAGNI principle - only add abstraction when patterns justify it

3. Staged Implementation (Zero-risk Adoption)¶

Phase 1: response_t → device_response_t (Response only)
Phase 2: Migrate all command handlers (no functional changes)
Phase 3: Unify event output (event_t → device_response_t)
Phase 4: Remove legacy code (response.h/cpp)

Benefit: Can roll back at any phase without breaking existing functionality

4. Backward Compatibility at Output Level¶

Client code written for v1.11.3 unified schema works unchanged in v1.12.0+
No breaking changes to firmware behavior
Only output format unified

What Changes, What Stays¶

Changes ✅¶

Component	Before	After
Response output	Flat or nested JSON	Unified schema JSON
Event output	SSV/TSV/CSV/JSONL (multi-format)	Unified schema JSON (JSONL)
API for handlers	`send_response(response_t)`	`send_device_response(device_response_t)`
Type system	`response_t` (union-based, 4-pair limit)	`device_response_t` (JsonDocument-based, unlimited)

Stays the Same ❌¶

Component	Status
event_t struct	Unchanged (still used internally by detection loop)
Command names/arguments	Unchanged (GET_VERSION, SET_THRESHOLD, etc.)
Hardware interfaces	Unchanged (serial comm, sensors, etc.)
Detection algorithm	Unchanged (cosmic_detector logic identical)
Configuration system	Unchanged (runtime_config.h/cpp)
SSV/TSV/CSV output	Unchanged (stream_formatter SSV/TSV/CSV paths remain)

Implementation Timeline¶

Phase	Scope	Duration	Target Version
1	Protocol layer + response_t replacement	1-2 days	v1.11.3
2	Command handler migration	2-3 weeks	v1.11.3 - v1.11.4
3	Event output unification	1-2 days	v1.12.0
4	Legacy code removal	1 day	v1.12.0+
Total	Full unification	~1 month	v1.12.0

Why This Approach?¶

Problem: Current State Fragmentation¶

response_t                      event_t
│                               │
├─ Builder pattern              ├─ Direct struct assignment
├─ Union-based (4-pair limit)   ├─ Flexible optional fields (#if)
├─ Always JSONL                 ├─ Multi-format (SSV/TSV/CSV/JSONL)
├─ Includes type/status         ├─ No type/status
└─ Command responses only       └─ Detection events only

Client (kazunoko)
├─ Expects type/status for responses
├─ Expects raw data for events
└─ Different parsing paths

Issue: Clients must maintain separate logic for response vs event

Solution: Unified Schema¶

device_response_t
├─ Type + Status + Data (all in unified format)
├─ No arbitrary limits (JsonDocument)
├─ Supports any nesting depth
├─ Conditional fields (ENABLE_*)
└─ Single output interface (send_device_response)

Client (any language)
├─ Single parser for all output
├─ Schema-based validation
├─ Language-specific code generation possible
└─ No special cases

Benefit: Symmetry and simplicity

Key Technical Decisions¶

Decision 1: JsonDocument vs Custom Struct¶

Chosen: JsonDocument (no Payload wrapper)

Aspect	JsonDocument	Custom Struct
Flexibility	⭐⭐⭐⭐⭐	⭐⭐
Code complexity	Simple	Moderate
Overhead	Minimal	Extra class
Nesting support	Unlimited	Manual
YAGNI compliance	✅	❌

Decision 2: Type + Status Location¶

Chosen: Inside JSON payload

// ✅ Chosen
{"type":"response","status":"ok","sent_at":123,"version":"1.10.0"}

// ❌ Not chosen (struct-level)
{"type":"response","status":"ok","payload":{"version":"1.10.0"}}

Rationale: Payload becomes the complete message (simpler serialization)

Decision 3: Timestamp Strategy¶

Chosen: Conditional device_get_timestamp()

uint32_t device_get_timestamp(void) {
  #if ENABLE_RTC
    return rtc_get_time();      // Unix time
  #else
    return millis() / 1000;     // Device uptime (seconds)
  #endif
}

Benefit: Unified timestamp regardless of RTC availability

Decision 4: Conditional Compilation (ENABLE_DEVICE_RESPONSE)¶

Phase 1-3: Optional flag (can disable for testing) Phase 4: Always enabled (flag removed after v1.12.0)

Benefit: Zero-risk adoption, easy rollback if issues arise

For detailed testing strategies, see: unified-device-response-protocol-design.md (Cross-Phase Validation section)

Client Integration (Future)¶

Once v1.12.0 is released, any client can implement:

# Python example
from unified_schema import DeviceResponse

def handle_osechi_message(jsonl_line):
    msg = DeviceResponse.model_validate_json(jsonl_line)

    if msg.type == "response":
        if msg.status == "ok":
            handle_command_response(msg)
        else:
            handle_error(msg.error_code, msg.error_message)

    elif msg.type == "event":
        process_detection_event(msg)

Language-agnostic:

Same code structure in JavaScript, Rust, Go, etc.
Automatic parser generation from JSON Schema
Zero ambiguity about message format

For risk assessment and mitigation details, see: unified-device-response-protocol-design.md (Cross-Phase Validation section)

For success criteria and deliverables checklist, see: unified-device-response-protocol-design.md

References¶

Executive Summary: 2025-12-08-device-response-1-executive-summary.md
Schema Specification: 2025-12-08-device-response-3-schema-specification.md
Implementation Checklist: 2025-12-08-device-response-4-implementation-checklist.md
Project REFACTORING_ROADMAP: kurikintons/REFACTORING_ROADMAP.md

Status: ✅ Architecture Complete, Ready for Phase 1 Implementation