Skip to content
  • Date Created: 2025-12-06
  • Last Modified: 2025-12-06

Progress Log: Response Schema Design for Serial Protocol

Task Description

Investigate the value and feasibility of introducing a formal response schema for the kurikintons serial protocol. Current context:

  • Firmware sends JSONL responses with "type":"response" and "status":"ok"|"error" headers
  • Data fields currently placed directly in response object (no wrapper)
  • Complex responses like GET_STATUS use nested JSON objects
  • No formal schema definition exists - documentation is text-based

Proposal: Introduce 3-layer response structure:

{
  "type": "response",
  "status": "ok"|"error",
  "payload": { /* command-specific data */ }
}

Key question: Is a formal schema (JSON Schema, TypeScript types, or documented) worth implementing?

Outcome

Analysis Results

  1. Current Response Architecture
  2. Type-safe response layer exists (response.h/cpp) using C++ union types
  3. Handles simple responses well: error, ok, single/multiple key-value pairs
  4. Does NOT handle nested structures - GET_STATUS uses raw JsonDocument

  5. Inconsistency: Some handlers use type-safe builders, others raw JSON

  6. Payload Wrapper Benefits (3-layer structure)

  7. ✓ Consistent structure across all responses
  8. ✓ Separates envelope (type/status) from content (payload)
  9. ✓ Flexible: payload can be nested or flat
  10. ✓ Client-side decision: flatten or preserve structure

  11. ✓ Future-proof: easier to add metadata to envelope

  12. Schema Definition Value

  13. ✓ Clarifies command response contracts
  14. ✓ Enables client-side type safety (TypeScript, JSON Schema validation)
  15. ✓ Single source of truth for API documentation
  16. ✓ Supports auto-generation of client stubs

  17. ⚠ Adds maintenance burden (schema must track implementation)

  18. Type Safety Analysis

  19. Current union-based approach: Compile-time checks via pair_count
  20. Proposed payload wrapper: Can maintain same safety with union variants
  21. Schema definition: Does NOT reduce type safety if done correctly
  22. Trade-off: Slightly increased JSON size (~40 bytes) for clarity

Design Decisions

Recommended approach: Phase-based adoption

Phase 1 (NOW): Document existing responses in CLAUDE.md

  • Add "Response Schema" section listing all GET_ and SET_ commands
  • Show example JSON for each command
  • No code changes required
  • Immediate value for client developers
  • Refactor response.h to support payload-based structure
  • All responses get "payload" wrapper
  • Maintains type safety via union variants
  • Backward compatible if done correctly (documentation bump to v2.0)

Phase 3 (Optional): Formal schema definition

  • JSON Schema (standards-based) OR TypeScript types (developer-friendly)
  • Place in /schemas/response-schema.json or /types/responses.ts
  • Optional client-side validation
  • Can be deferred to v2.0+ cycle

Current Response Examples

Simple response (GET_VERSION):

{"type":"response","status":"ok","version":"1.10.6"}

Complex response (GET_STATUS):

{
  "type":"response","status":"ok",
  "system":{"version":"1.10.6","uptime_ms":12345,"adc_channel":32},
  "detection":{"poll_count":100,"deadtime_ms":0,"threshold1":1234},
  "features":{"bme280":1,"rtc":1,"timestamp":1,"wifi":0}
}

Error response:

{"type":"response","status":"error","error_code":1}

Learnings

  1. Schema is valuable, not just academic
  2. Current state: Developers must read source code or docs to understand response format
  3. With schema: Machine-readable contract between firmware and client

  4. ESP32 environment: No excuse for poor API contracts (unlike resource-constrained IoT)

  5. Payload wrapper aligns with REST conventions

  6. Separates metadata (type/status) from data (payload)
  7. Similar to HTTP: status code + response body structure

  8. Makes error handling uniform across clients

  9. Type safety maintained through careful design

  10. Union-based variant dispatch still works with payload wrapper
  11. pair_count discriminator can be extended to handle nested variants

  12. No requirement to sacrifice type checking for clarity

  13. Implementation debt already exists

  14. GET_STATUS is inconsistent (raw JsonDocument vs type-safe builders)
  15. TEST_LED handler uses raw JSON when it could use builders
  16. Inconsistency makes maintaining the protocol harder over time

Next Steps

Immediate (this session)

  • Add "Response Schema" section to CLAUDE.md
  • Document all command responses with examples
  • Update README or getting-started guide with schema reference

Short-term (v1.11.x cycle)

  • Evaluate payload wrapper feasibility
  • Create migration plan for existing handlers
  • Decide on schema format (JSON Schema vs TypeScript)

Medium-term (v2.0 cycle)

  • Implement payload wrapper if Phase 1 is successful
  • Refactor response.h to support nested variants
  • Deploy formal schema (JSON Schema or TS types)
  • Generate client stub code from schema (optional)