Unified Device Response Protocol - Executive Summary¶

Prepared for: qumasan Date: 2025-12-08 Status: Ready for Implementation

Quick Links:

Full Schema Spec: 2025-12-08-device-response-3-schema-specification.md
Architecture Overview: 2025-12-08-device-response-2-architecture-overview.md
Implementation Checklist: 2025-12-08-device-response-4-implementation-checklist.md

What We Designed¶

Unified Device Response Protocol - A single JSON schema for all OSECHI output (responses + events).

Core Schema¶

{
  "type": "response" | "event",
  "status": "ok" | "error",
  "sent_at": 1732046789,
  "<payload>": <data>
}

Impact:

✅ Clients can use single parser regardless of message type
✅ Any language can implement client (schema is language-neutral)
✅ No breaking changes to firmware behavior

How We Get There (4 Phases)¶

Phase	What	When	Lines of Code
1	Create `device_response_t` + migrate response_t	v1.12.0	+150 lines
2	Migrate all 25+ command handlers	v1.12.0-1.12.1	~300 lines changed
3	Unify event output (stream_formatter)	v1.12.0+	~100 lines changed
4	Delete response.h/cpp (cleanup)	v1.13.0+	-200 lines removed

Total Duration: ~1 month Risk Level: Low (staged implementation, easy rollback)

Key Design Decisions¶

✅ JsonDocument Direct (No Payload Wrapper)¶

Why: YAGNI principle - avoid premature abstraction

JsonDocument is already flexible enough
Payload wrapper would add 60+ lines without benefit
Can add helpers later if patterns emerge

✅ Payload Contains type/status/sent_at¶

Why: Simpler serialization

// ✅ Chosen (simpler)
JsonDocument doc;
doc["type"] = "response";
doc["status"] = "ok";
doc["sent_at"] = device_get_timestamp();
doc["version"] = "1.10.0";
send_device_response({std::move(doc)});

// ❌ Alternative (nested structure complexity)
JsonDocument doc;
doc["payload"]["version"] = "1.10.0";
// ... more boilerplate

✅ Single device_response_t for Response + Event¶

Why: Symmetry and single implementation path

Response: type="response", status="ok"|"error"
Event: type="event", status="ok"
Client code: Single parser for both

✅ Conditional Compilation (ENABLE_DEVICE_RESPONSE)¶

Why: Zero-risk adoption

Phase 1: Optional (can test both old + new in parallel)
Phase 4: Always enabled (flag removed after full migration)
Easy rollback if issues found

What Changes, What Stays Same¶

Changes (Output Level Only)¶

Before	After
`{"type":"response","status":"ok","version":"1.10.0"}`	`{"type":"response","status":"ok","sent_at":1732046789,"version":"1.10.0"}`
`{"hit1":95,"hit2":87,...}` (events, SSV format)	`{"type":"event","status":"ok","sent_at":1732046789,"hit1":95,"hit2":87,...}`
Multiple output formats (SSV/TSV/CSV/JSONL)	Single unified JSONL format
Separate parsing for response vs event	Single unified parser

Stays the Same (Firmware Behavior)¶

✅ Command names (GET_VERSION, SET_THRESHOLD, etc.) unchanged
✅ Detection algorithm identical
✅ Hardware interfaces identical
✅ Event generation rate unchanged
✅ Command latency unchanged

Implementation Approach¶

Phase 1: Create Protocol Layer (1-2 days)¶

// src/device_response.h
typedef struct {
  JsonDocument payload;  // Contains: type, status, sent_at, data
} device_response_t;

// Builders
device_response_t device_response_ok(const char* key, int32_t value);
device_response_t device_response_error(int code, const char* msg);

// Serialization
void send_device_response(const device_response_t& resp);

Phase 2: Migrate Handlers (2-3 weeks)¶

Replace in text_command_manager.cpp:

// Before
send_response(response_string("version", config_get_version()));

// After
send_device_response(device_response_ok("version", config_get_version()));

Do this for ~25 command handlers, grouped by complexity.

Phase 3: Event Output (1-2 days)¶

Modify stream_formatter.cpp to output unified schema:

{"type":"event","status":"ok","sent_at":123456,"hit1":95,"hit2":87,...}

Phase 4: Cleanup (1 day)¶

Delete response.h/cpp (legacy code removal).

Client Integration (Future)¶

Once v1.12.0 is released:

# Any language (Python example)
from unified_schema import DeviceResponse

def process_osechi_output(jsonl_line):
    msg = DeviceResponse.model_validate_json(jsonl_line)

    # Single parser handles both response + event
    if msg.type == "response":
        handle_command_response(msg)
    elif msg.type == "event":
        handle_detection_event(msg)

Multi-language Support: Same code structure in JS, Rust, Go, etc.

Why This Matters¶

Before (Current State)¶

OSECHI ──> response_t ─────┐
                            ├──> Separate client parsers
OSECHI ──> event_t ────────┘    (response vs event logic)

Client must handle:

Different JSON structures
Different timestamp semantics
Special cases per message type

After (v1.12.0+)¶

OSECHI ──> device_response_t (unified) ──> Serial
                                              │
                                              ▼
                          Single Client Parser
                          (response + event)
                                              │
                                ┌─────────────┼──────────────┐
                                ▼             ▼              ▼
                            kazunoko     JavaScript Client   Rust Client

Benefit: Symmetry, simplicity, extensibility

Risk Assessment¶

Risk	Impact	Mitigation
Bugs in protocol layer	Medium	Phase 1 testing in parallel (ENABLE_DEVICE_RESPONSE flag)
Incomplete handler migration	High	Automated CI/CD checks (grep for remaining send_response)
Memory exhaustion (large JSON)	Low	Stress test (1000 events, all ENABLE flags)
Timestamp inconsistency	Low	Clear documentation + client handles both formats

Overall: Low Risk (staged approach + easy rollback at each phase)

Deliverables Prepared¶

Design Documents (Ready ✅)¶

unified-device-response-schema.md - Complete schema specification
implementation-checklist.md - Detailed task-by-task breakdown
project-overview.md - Holistic view of initiative
REFACTORING_ROADMAP.md - Updated with new section

Code Changes (Planned)¶

src/device_response.h/cpp - New protocol layer
text_command_manager.cpp - Handler migration
stream_formatter.cpp - Event output unification
config.h - ENABLE_DEVICE_RESPONSE flag

Documentation (Planned)¶

Schema spec (JSON Schema format)
Client integration guide
Migration guide
Updated CLAUDE.md

Timeline¶

Week 1 (Dec 9-13): Phase 1 (Protocol layer) v1.12.0
Week 2-4 (Dec 16-Jan 3): Phase 2 (Handler migration) v1.12.0-1.12.1
Week 5 (Jan 6-10): Phase 3 (Event unification) v1.12.0+
Week 5 (Jan 6-10): Phase 4 (Legacy cleanup) v1.13.0+

Total: ~5 weeks (including testing + documentation)

Success Criteria (v1.12.0)¶

✅ All OSECHI output follows unified schema
✅ Single schema definition in docs/schemas/device-response.json
✅ All client libraries (kazunoko, etc.) updated to unified parser
✅ No behavioral changes to firmware (backward compatible)
✅ Binary size reduced 5-10KB
✅ Documentation complete

Next Action¶

Ready to start Phase 1?

Approve this design
Create feature branch: feature/device-response-protocol
Implement src/device_response.h/cpp
Begin Phase 2 migration

Questions Answered¶

Q: Why not just extend response_t? A: response_t is union-based with 4-pair limit. JsonDocument is more flexible.

Q: Why Payload wrapper? A: Avoided (YAGNI). JsonDocument is sufficient.

Q: sent_at inside or outside payload? A: Inside (simpler serialization).

Q: event_t vs response_t to device_response_t? A: Both eventually unified (response first, events in Phase 3).

Q: Multi-language client support? A: Yes - schema is fixed, so any language can implement parser.

Key Insight¶

A single schema definition enables client implementations across any language.

Once v1.12.0+ is released, clients need only understand one JSON structure. This eliminates special cases, reduces code duplication, and enables auto-generated parsers from the schema.

Design Status: ✅ Complete and approved Implementation Status: Ready to begin Owner: Shota Takahashi

Start Date: 2025-12-09 (Phase 1) Target Completion: 2026-01-10 (v1.12.0)