Skip to content
  • Date Created: 2025-12-10
  • Last Modified: 2025-12-11 (Updated: Phase 6 implementation completed)

Progress Log: DeviceResponse Class Design → Phase 6 Implementation (v1.13.12)

Overview

This document defines the design for Phase 0.5: implementation of the DeviceResponse class for unified Layer 2 (conversion) and Layer 3 (serialization) functionality within the unified device response protocol (ENABLE_DEVICE_RESPONSE=1).

Key Premise: When ENABLE_DEVICE_RESPONSE=1, the firmware uses a clean 3-layer architecture for all output (commands and events). Legacy response handling is completely disabled.

Architecture: Three-Layer Data Flow

Complete Data Pipeline

┌─────────────────────────────────────────────────────────────────────────┐
│ INPUT: User Command or Cosmic Ray Detection                             │
└─────────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────────┐
│ PARSING: CommandQueue or EventQueue                                      │
│  - Receive: Serial input → buffer                                        │
│  - Parse: buffer → command_t / event_t (domain-specific structures)      │
│  - Dispatch: command_t/event_t → handler function                        │
└─────────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────────┐
│ LAYER 1: Handler Logic & Validation                                      │
│  - Input: command_t (parsed command) / event_t (sensor data)             │
│  - Handler: Execute logic, validate arguments, handle errors             │
│  - Output: command_response_t / event_response_t (validation metadata)   │
│                                                                           │
│  Example (command handler - success):                                    │
│    - Input: command_t {name="GET_VERSION", arg_count=0}                  │
│    - Logic: Call Command::getInstance().get_version()                    │
│    - Output: command_response_t {is_ok=true, status=OK, error_code=0}   │
│                                                                           │
│  Example (command handler - error):                                      │
│    - Input: command_t {name="SET_THRESHOLD", arg_count=1, args[0]="9999"} │
│    - Logic: Validate range [0-1023], parse argument                      │
│    - Output: command_response_t {                                        │
│        is_ok=false,                                                      │
│        status=ERROR,                                                     │
│        error_code=DEVICE_CODE_OUT_OF_RANGE (2),                          │
│        error_message="Value 9999 out of range [0-1023]"                   │
│      }                                                                     │
│                                                                           │
│  Example (event validation - success):                                   │
│    - Input: event_t {hit1=50, hit2=45, hit3=40, adc=2048}               │
│    - Logic: Validate sensor data (all channels > 0)                      │
│    - Output: event_response_t {is_ok=true, status=OK, error_code=0}     │
│                                                                           │
│  Example (event validation - error):                                     │
│    - Input: event_t {hit1=0, hit2=0, hit3=0, adc=0}                     │
│    - Logic: Sensor validation fails (no active channels)                 │
│    - Output: event_response_t {                                          │
│        is_ok=false,                                                      │
│        status=ERROR,                                                     │
│        error_code=DEVICE_CODE_NOT_READY (3),                             │
│        error_message="Sensors not ready: no channels active"             │
│      }                                                                     │
└─────────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────────┐
│ LAYER 2: DeviceResponse Conversion (NEW)                                 │
│  - Input: command_response_t / event_response_t (Layer 1 output)         │
│  - Responsibility: Convert to transport envelope (device_response_t)     │
│  - Output: device_response_t (unified envelope, no JSON yet)             │
│                                                                           │
│  Method: DeviceResponse::from_command(command_response_t)                │
│  Method: DeviceResponse::from_event(event_response_t)                    │
│                                                                           │
│  - Copies all status/error fields from Layer 1                           │
│  - Adds type field (RESPONSE or EVENT)                                   │
│  - NO JSON serialization at this layer                                   │
└─────────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────────┐
│ LAYER 3: DeviceResponse Serialization & Transmission (NEW)               │
│  - Input: device_response_t (fully populated envelope + payload)         │
│  - Responsibility: Serialize to JSON and send to Serial                  │
│  - Output: JSON Lines (JSONL) to Serial port                             │
│                                                                           │
│  Method: DeviceResponse::send(device_response_t)                         │
│                                                                           │
│  - Serializes all fields from device_response_t                          │
│  - No conditional logic (all payload assembly done in Layer 2→3)         │
│  - Output format: Complete JSON on single line                           │
│                                                                           │
│  Example output (command - success):                                     │
│    {"type":"response","status":"ok","sent_at":12345,"version":"v1.14.0"}│
│                                                                           │
│  Example output (command - error):                                       │
│    {"type":"response","status":"error","sent_at":12345,                  │
│     "error_code":2,"error_message":"Value 999 out of range [0-1023]"}   │
│                                                                           │
│  Example output (event - success):                                       │
│    {"type":"event","status":"ok","sent_at":12345,                        │
│     "hit1":50,"hit2":45,"hit3":40,"adc":2048}                            │
│                                                                           │
│  Example output (event - error):                                         │
│    {"type":"event","status":"error","sent_at":12345,                     │
│     "error_code":3,"error_message":"Sensors not ready: no channels active"}│
└─────────────────────────────────────────────────────────────────────────┘
                                ↓
                          Serial Output

Layer 1: Validation Response Types

Error Code Reference

All errors use standardized error codes from device_response_code_t enum defined in include/device_response.h:95-102.

Both command_response_t and event_response_t use the same error codes for consistency:

Code Enum Meaning Example
0 DEVICE_CODE_OK Success (no error) Command executed successfully
1 DEVICE_CODE_INVALID_ARG Invalid argument Missing required argument, wrong type
2 DEVICE_CODE_OUT_OF_RANGE Value out of range Threshold 2000 when max is 1023
3 DEVICE_CODE_NOT_READY Device not ready Sensors not active, no signal
4 DEVICE_CODE_TIMEOUT Operation timeout Command took too long
5 DEVICE_CODE_UNKNOWN Unknown error Unexpected internal error

Source Definition (include/device_response.h):

typedef enum {
  DEVICE_CODE_OK = 0,
  DEVICE_CODE_INVALID_ARG = 1,
  DEVICE_CODE_OUT_OF_RANGE = 2,
  DEVICE_CODE_NOT_READY = 3,
  DEVICE_CODE_TIMEOUT = 4,
  DEVICE_CODE_UNKNOWN = 5
} device_response_code_t;

command_response_t (Phase 0.5 Update Required)

Current Location: include/command_queue.h:140-161 (legacy definition)

Current (obsolete for Phase 0.5):

typedef struct {
  bool is_ok;              // Handler execution success
  char message[512];       // JSON response (will be replaced)
} command_response_t;

Phase 0.5 Definition (required update):

typedef struct {
  bool is_ok;              // Handler execution success
  device_response_status_t status;   // OK or ERROR
  device_response_code_t error_code; // Error classification (0-5)
  char error_message[256]; // Human-readable error
  uint32_t sent_at;        // Timestamp from dispatcher
  device_response_type_t type; // RESPONSE (for commands)
} command_response_t;

Key Changes:

  • Remove legacy message[512] field (no JSON at Layer 1)
  • Add status (from device_response.h: device_response_status_t)
  • Add error_code (from device_response.h: device_response_code_t with enum 0-5)
  • Add error_message[256] (structured error info)
  • Add sent_at (timestamp added by dispatcher)
  • Add type (always DEVICE_TYPE_RESPONSE for commands)
  • Requires: #include "device_response.h" in command_queue.h

Responsibility: Handler returns this after executing command logic

  • Validation of arguments
  • Error handling (invalid args, out of range, not ready)
  • Status and error information (no JSON construction)

event_response_t (Phase 0.5 Update Required)

Current Location: include/event_queue.h:150-220 (to be created with Phase 0.5)

Phase 0.5 Definition:

typedef struct {
  bool is_ok;              // Sensor validation success
  device_response_status_t status;   // OK or ERROR
  device_response_code_t error_code; // Error classification (0-5)
  char error_message[256]; // Human-readable error
  uint32_t sent_at;        // Timestamp from validator
  device_response_type_t type; // EVENT (for events)
} event_response_t;

Key Characteristics:

  • Identical structure to command_response_t (only type field differs)
  • Uses unified error codes from device_response.h
  • Requires: #include "device_response.h"

Responsibility: Sensor validation returns this after checking event data

  • Sensor data validation (all channels working)
  • Error handling (sensor not ready, invalid data)
  • Status and error information (no sensor data copied here)

Layer 2: device_response_t (Unified Transport Structure)

Location: include/device_response.h:104-121

Current Structure (INCORRECT - envelope only)

typedef struct {
  device_response_type_t type;       // RESPONSE or EVENT
  device_response_status_t status;   // OK or ERROR
  uint32_t sent_at;                  // Timestamp in seconds
  device_response_code_t error_code; // Error code (0-5)
  char error_message[256];           // Error description
  // Payload: NOT included in struct (added dynamically in Layer 3)
} device_response_t;

Problem with current design: - Only stores envelope fields (metadata) - Payload fields are added dynamically via ArduinoJson in Layer 3 - Violates clean separation: Layer 1 (handler) populates payload in command_response_t, but Layer 2 doesn't carry it forward - Requires dispatcher or send() to know about command-specific fields (type-specific conditional logic)

Corrected Structure (FOLLOWING JSON SCHEMA)

typedef struct {
  // Envelope fields (always present)
  device_response_type_t type;       // RESPONSE or EVENT
  device_response_status_t status;   // OK or ERROR
  uint32_t sent_at;                  // Timestamp in seconds

  // Error fields (present only when status=ERROR)
  device_response_code_t error_code; // Error code (1-5)
  char error_message[256];           // Human-readable error (max 255 chars)

  // Payload fields (command or event specific)
  // The struct must accommodate ALL possible fields from device-response.json

  // Command response fields (when type=RESPONSE)
  char version[32];                  // Firmware version (GET_VERSION)
  uint32_t uptime_ms;                // Device uptime (GET_STATUS, GET_UPTIME)
  char mac_address[18];              // MAC address (GET_MAC_ADDRESS, GET_STATUS)
  uint16_t poll_count;               // Detection poll count (GET_STATUS)
  uint16_t deadtime_ms;              // Detector deadtime (GET_STATUS)
  struct {                           // Threshold config (GET_THRESHOLD)
    uint8_t channel;
    uint16_t value;
  } threshold;
  // ... additional command response fields (system, detection, features)

  // Event fields (when type=EVENT)
  uint8_t hit1, hit2, hit3;          // Detection counts (0-100)
  uint16_t adc;                      // ADC raw value (0-4095)
  float tmp_c;                       // Temperature (ENABLE_BME280)
  float atm_pa;                      // Atmospheric pressure (ENABLE_BME280)
  float hmd_pct;                     // Humidity (ENABLE_BME280)
  uint32_t uptime_ms;                // Uptime (ENABLE_TIMESTAMP)
  uint64_t timedelta_us;             // Inter-event duration (ENABLE_TIMESTAMP)
  uint32_t unix_timestamp;           // Absolute time (ENABLE_RTC)
  uint8_t hit_type;                  // Channel bitmask (ENABLE_HITTYPE)
  // ... additional event fields (gnss)
} device_response_t;

Key Insight:

  • device_response_t is the complete transport structure, not just an "envelope"
  • Follows JSON Schema structure from docs/schemas/device-response.json
  • Combines metadata (type, status, sent_at, error fields) with all possible payload fields
  • Flexible design: Only populated fields are serialized by send() to JSON

Memory Analysis: Flat vs. Union Design

Current Flat Structure Size (Approximate):

Envelope fields:
  type:           1 byte (enum)
  status:         1 byte (enum)
  sent_at:        4 bytes (uint32_t)
  error_code:     1 byte (enum)
  error_message:  256 bytes (char[256])
Subtotal:         263 bytes

Command response fields:
  version:        32 bytes (char[32])
  uptime_ms:      4 bytes (uint32_t)
  mac_address:    18 bytes (char[18])
  poll_count:     2 bytes (uint16_t)
  deadtime_ms:    2 bytes (uint16_t)
  threshold:      3 bytes (struct: uint8_t + padding + uint16_t)
Subtotal:         61 bytes

Event fields:
  hit1, hit2, hit3: 3 bytes (uint8_t × 3)
  adc:            2 bytes (uint16_t)
  tmp_c:          4 bytes (float)
  atm_pa:         4 bytes (float)
  hmd_pct:        4 bytes (float)
  uptime_ms:      4 bytes (uint32_t) [duplicate name, different field]
  timedelta_us:   8 bytes (uint64_t)
  unix_timestamp: 4 bytes (uint32_t)
  hit_type:       1 byte (uint8_t)
Subtotal:         38 bytes

Total flat structure: ~362 bytes

Proposed Union Structure Size:

// Note: Payload is union of command OR event (never both)
typedef struct {
  // Envelope (always 263 bytes)
  device_response_type_t type;
  device_response_status_t status;
  uint32_t sent_at;
  device_response_code_t error_code;
  char error_message[256];

  // Payload (union: max(61, 38) = 61 bytes)
  union {
    command_payload_t command;  // 61 bytes
    event_payload_t event;      // 38 bytes
  } payload;

} device_response_t;

Total union structure: 263 + 61 = 324 bytes (saves 38 bytes)

Memory Savings Analysis:

  • Flat design: 362 bytes per message
  • Union design: 324 bytes per message
  • Savings: 38 bytes per message (~10% reduction)
  • Context: ESP32 has 320KB total RAM; savings significant only if buffering many messages

Union Pattern Evaluation

PROS:

  • ✅ Clear semantic separation: union shows "command OR event, never both"
  • ✅ 10% memory savings (38 bytes per message)
  • ✅ Explicit intent in code structure
  • ✅ Prevents accidental field pollution (no command fields in event payload)
  • ✅ Easier to reason about valid fields at any time

CONS:

  • ❌ More complex struct definition (union nesting)
  • ❌ Requires explicit payload access: response.payload.command.version
  • ❌ send() method needs to know union member names
  • ❌ Type conversions in from_command()/from_event() more complex
  • ❌ Harder to debug (union fields not visible in debugger without knowing member)
  • ❌ Marginal benefit for ESP32 (38 bytes in 320KB = 0.01% of RAM)

Recommendation: Flat Design (Current Approach)

Why flat design is preferred here:

  1. Simplicity over marginal savings: 38 bytes is negligible on ESP32 (320KB available)
  2. Debuggability: All fields visible at once without union complexity
  3. Performance: No indirection layer (union access adds one level)
  4. Handler clarity: Handlers directly assign fields without union syntax
  5. Serial output: send() can serialize all fields uniformly
  6. Code readability: response.version vs response.payload.command.version

When Union Would Be Better:

  • Memory severely constrained (<16KB RAM) → union saves meaningful percentage
  • Many messages buffered simultaneously (command queue depth > 100)
  • Strict field validation required (union prevents cross-type pollution)

Conclusion: For this firmware (ESP32 with 320KB RAM, single-message processing), flat design is optimal. The 10% savings don't justify the complexity trade-offs. Union is over-engineering for this use case.

If future memory constraints emerge (e.g., porting to STM32L0 with 8KB RAM), union refactoring becomes a simple, localized change

Responsibility: Complete transport structure for all device output

  • Carries all metadata needed for JSON output (type, status, sent_at)
  • Carries all payload fields populated by handler (Layer 1 output)
  • Ready for direct serialization without further transformation
  • Single structure for all command/event variants

Phase 0.5 Design: DeviceResponse Class

Design Principle

The DeviceResponse class is a static utility class that:

  1. Converts Layer 1 responses → Layer 2 envelope
  2. Serializes Layer 2 envelope → JSON + Serial transmission

No instantiation. All methods are static and pure functions (except Serial I/O side effect).

Class Signature

/**
 * @brief DeviceResponse - Unified Layer 2→3 pipeline for all device output
 *
 * Converts Layer 1 response types (command_response_t, event_response_t)
 * to Layer 2 device_response_t envelope, and serializes to JSON Lines format.
 *
 * Design: Static utility class (no instantiation)
 * Thread-safe: All methods are side-effect-free except Serial output
 * Memory-efficient: No heap allocation, direct serial streaming
 * Conditional compilation: Only compiled when ENABLE_DEVICE_RESPONSE=1
 */
class DeviceResponse {
 public:
  /**
   * @brief Convert command handler result → device response (complete)
   *
   * Layer 1→2 conversion for command responses.
   *
   * Takes the result returned by a command handler (Layer 1, which includes
   * status/error fields AND command-specific payload) and converts it to a
   * fully-populated device_response_t ready for serialization (Layer 2).
   *
   * @param response Command handler result (Layer 1: command_response_t with payload)
   * @return device_response_t Complete response with all fields populated by handler
   *
   * @note Pure function: No side effects, deterministic output
   * @note Called by: CommandQueue::dispatch() after handler execution
   * @note Input invariant: handler_result must include all payload fields (version, etc.)
   * @note Output invariant: returned device_response_t ready for serialization (no further assembly needed)
   * @note Next step: Call DeviceResponse::send() with no modifications
   *
   * Example usage (CORRECT flow):
   *   // Layer 1: Handler populates command_response_t with status AND payload
   *   command_response_t handler_result = handle_version(cmd);
   *   // handler_result.status = OK
   *   // handler_result.version = "v1.14.0"   ← populated by handler
   *   // handler_result.uptime_ms = millis()  ← populated by handler
   *
   *   // Layer 2: Convert to device_response_t (copy all fields)
   *   device_response_t response = DeviceResponse::from_command(handler_result);
   *   // response now contains: type, status, sent_at, version, uptime_ms, etc.
   *
   *   // Layer 3: Serialize (no modification needed)
   *   DeviceResponse::send(response);
   *   // Output: {"type":"response","status":"ok","sent_at":123,"version":"v1.14.0","uptime_ms":1234}
   */
  static device_response_t from_command(const command_response_t& response);

  /**
   * @brief Convert event validation result → device response (complete)
   *
   * Layer 1→2 conversion for event responses.
   *
   * Takes the result from event validation (Layer 1, which includes
   * status/error fields AND event-specific payload) and converts it to a
   * fully-populated device_response_t ready for serialization (Layer 2).
   *
   * @param response Event validation result (Layer 1: event_response_t with payload)
   * @return device_response_t Complete response with all fields populated by validator
   *
   * @note Pure function: No side effects, deterministic output
   * @note Called by: Detection loop in main.cpp after event_to_response()
   * @note Input invariant: validation result must include all payload fields (hit1, hit2, hit3, adc, etc.)
   * @note Output invariant: returned device_response_t ready for serialization (no further assembly needed)
   * @note Next step: Call DeviceResponse::send() with no modifications
   *
   * Example usage (CORRECT flow):
   *   // Layer 1: Validator populates event_response_t with status AND payload
   *   event_response_t validation = event_to_response(&event);
   *   // validation.status = OK or ERROR
   *   // validation.hit1 = event.hit1       ← populated by validator
   *   // validation.hit2 = event.hit2       ← populated by validator
   *   // validation.hit3 = event.hit3       ← populated by validator
   *   // validation.adc = event.adc         ← populated by validator
   *
   *   // Layer 2: Convert to device_response_t (copy all fields)
   *   device_response_t response = DeviceResponse::from_event(validation);
   *   // response now contains: type, status, sent_at, hit1, hit2, hit3, adc, etc.
   *
   *   // Layer 3: Serialize (no modification needed)
   *   DeviceResponse::send(response);
   *   // Output: {"type":"event","status":"ok","sent_at":12345,"hit1":50,"hit2":45,"hit3":40,"adc":2048}
   */
  static device_response_t from_event(const event_response_t& response);

  /**
   * @brief Serialize device response → JSON Lines to Serial
   *
   * Layer 3 serialization: universal for both commands and events.
   *
   * Takes a fully-populated device_response_t (with all fields already populated
   * by handler/validator in Layer 1) and serializes to JSON Lines format on Serial.
   *
   * @param response Fully-populated device response (Layer 2: all fields ready)
   * @return void (output goes to Serial)
   *
   * @note Serializes as single JSON object on one line (JSONL format)
   * @note Input invariant: response must be fully populated; no assembly done in this method
   * @note Called by: CommandQueue::dispatch() or detection loop in main.cpp
   *
   * Serialization rules:
   *   - Always includes: type, status, sent_at (envelope fields)
   *   - When status="error": includes error_code and error_message
   *   - When status="ok": includes all populated payload fields from device_response_t
   *   - All fields serialized as-is from device_response_t (no conditional logic)
   *
   * Example usage (command - success):
   *   command_response_t result = handler(cmd);      // Layer 1: handler populates version, uptime_ms
   *   device_response_t response = DeviceResponse::from_command(result);  // Layer 2: copy all fields
   *   DeviceResponse::send(response);                 // Layer 3: serialize as-is
   *   // Output: {"type":"response","status":"ok","sent_at":123,"version":"v1.14.0","uptime_ms":1234}
   *
   * Example usage (event - success):
   *   event_response_t validation = event_to_response(&event);  // Layer 1: validator populates hit1, hit2, hit3, adc
   *   device_response_t response = DeviceResponse::from_event(validation);  // Layer 2: copy all fields
   *   DeviceResponse::send(response);                           // Layer 3: serialize as-is
   *   // Output: {"type":"event","status":"ok","sent_at":12345,"hit1":50,"hit2":45,"hit3":40,"adc":2048}
   *
   * Example usage (error):
   *   command_response_t result = handler(cmd);      // Layer 1: returns status=ERROR with error_code, error_message
   *   device_response_t response = DeviceResponse::from_command(result);  // Layer 2: copy all fields
   *   DeviceResponse::send(response);                 // Layer 3: serialize error only (no payload)
   *   // Output: {"type":"response","status":"error","sent_at":123,"error_code":1,"error_message":"Invalid argument"}
   */
  static void send(const device_response_t& response);

 private:
  // Internal helper functions (not part of public API)
  // Implementation details only
};

Implementation Details

Layer 2→3 Conversion: from_command()

device_response_t DeviceResponse::from_command(const command_response_t& response) {
  device_response_t result;

  // Copy all Layer 1 fields to Layer 2
  result.type = response.type;        // DEVICE_TYPE_RESPONSE
  result.status = response.status;    // DEVICE_STATUS_OK or _ERROR
  result.sent_at = response.sent_at;
  result.error_code = response.error_code;

  // Copy error message
  strncpy(result.error_message, response.error_message,
          sizeof(result.error_message) - 1);
  result.error_message[sizeof(result.error_message) - 1] = '\0';

  return result;
}

Invariants:

  • No JSON construction at this layer
  • Pure data structure copy
  • All fields copied as-is

Layer 2→3 Conversion: from_event()

device_response_t DeviceResponse::from_event(const event_response_t& response) {
  device_response_t result;

  // Copy all Layer 1 fields to Layer 2
  result.type = response.type;        // DEVICE_TYPE_EVENT
  result.status = response.status;    // DEVICE_STATUS_OK or _ERROR
  result.sent_at = response.sent_at;
  result.error_code = response.error_code;

  // Copy error message
  strncpy(result.error_message, response.error_message,
          sizeof(result.error_message) - 1);
  result.error_message[sizeof(result.error_message) - 1] = '\0';

  return result;
}

Invariants:

  • No JSON construction at this layer
  • Pure data structure copy (only status/error fields)
  • Sensor data payload assembly done in Layer 2→3 (before send() is called)

Layer 3 Serialization: send()

void DeviceResponse::send(const device_response_t& response) {
  StaticJsonDocument<512> doc;

  // 1. Add envelope fields (always present)
  doc["type"] = (response.type == DEVICE_TYPE_RESPONSE) ? "response" : "event";
  doc["status"] = (response.status == DEVICE_STATUS_OK) ? "ok" : "error";
  doc["sent_at"] = response.sent_at;

  // 2. Add error fields (only when status="error")
  if (response.status == DEVICE_STATUS_ERROR) {
    doc["error_code"] = response.error_code;
    doc["error_message"] = response.error_message;
  }

  // 3. Serialize all fields from device_response_t
  // Note: Payload fields are already in response struct, populated in Layer 2→3
  // No conditional logic here - all assembly done upstream
  serializeJson(doc, Serial);
  Serial.println();  // Newline for JSONL format
}

Design Principle:

  • Universal method for both commands and events
  • Payload assembly responsibility in caller (CommandQueue or main.cpp)
  • Layer 3 only serializes - no domain-specific logic
  • All conditional field handling done in Layer 2→3 (preprocessor directives in caller)

Integration with CommandQueue and Detection Loop

CommandQueue::dispatch() Integration

Current Flow (before Phase 0.5):

handler(cmd) → command_response_t → Serial.println(resp.message)

After Phase 0.5 (CORRECT FLOW):

handler(cmd) → command_response_t with payload (Layer 1)
              ↓
DeviceResponse::from_command() → device_response_t with payload (Layer 2)
              ↓
DeviceResponse::send() → JSON to Serial (Layer 3, no assembly needed)

Implementation in CommandQueue::dispatch():

static command_response_t dispatch(const command_t& cmd) {
  // 1. Find handler in command table
  command_handler_t handler = find_handler(cmd.name);
  if (!handler) {
    return device_response_error(DEVICE_TYPE_RESPONSE,
                                 DEVICE_CODE_UNKNOWN, "Unknown command");
  }

  // 2. Execute handler (Layer 1)
  // Handler populates BOTH status/error fields AND payload fields
  command_response_t handler_result = handler(cmd);
  // Example: handler_result.version = "v1.14.0"
  //          handler_result.uptime_ms = millis()
  //          handler_result.status = DEVICE_STATUS_OK

  // 3. Add dispatcher context (sent_at timestamp)
  handler_result.sent_at = device_get_timestamp();

  // 4. Convert to device_response_t (Layer 2)
  // All fields (status, error, payload) copied as-is
  device_response_t response = DeviceResponse::from_command(handler_result);
  // response now contains all fields from handler_result

  // 5. Layer 3: Serialize to JSON and send (NO modifications needed)
  // send() serializes all populated fields
  DeviceResponse::send(response);

  return handler_result;
}

Key Design Principle:

  • Layer 1 (handler): Populates command_response_t with status/error AND payload (version, uptime_ms, etc.)
  • Layer 2 (from_command): Copies all fields to device_response_t (pure data transfer)
  • Layer 3 (send): Serializes fully-populated device_response_t as-is (no conditional logic)

Detection Loop Integration (main.cpp)

Current Flow (before Phase 0.5):

detect_event() → event_t → Manual JSON construction → Serial output

After Phase 0.5 (CORRECT FLOW):

detect_event() → event_t (sensor data)
              ↓
event_to_response() → event_response_t with payload (Layer 1: validation + population)
              ↓
DeviceResponse::from_event() → device_response_t with payload (Layer 2)
              ↓
DeviceResponse::send() → JSON to Serial (Layer 3, no assembly needed)

Implementation in main.cpp detection loop:

void loop() {
  // ... detection logic ...

  if (detection.detected) {
    // 1. Build event_t from sensor data
    event_t event;
    event.hit1 = detection.channel1;
    event.hit2 = detection.channel2;
    event.hit3 = detection.channel3;
    event.sensorValue = adc_read();
#if ENABLE_TIMESTAMP
    event.uptime_ms = millis();
    event.timedelta_us = micros() - last_event_time_us;
    last_event_time_us = micros();
#endif
#if ENABLE_BME280
    event.tmp_c = bme280_read_temperature();
    event.atm_pa = bme280_read_pressure();
    event.hmd_pct = bme280_read_humidity();
#endif
    // ... populate optional fields ...

    // 2. Validate and populate event_response_t (Layer 1)
    // Validator populates BOTH status/error fields AND payload fields
    event_response_t validation = event_to_response(&event);
    // validation.hit1 = event.hit1
    // validation.hit2 = event.hit2
    // validation.hit3 = event.hit3
    // validation.adc = event.sensorValue
    // validation.status = DEVICE_STATUS_OK (if validation passed)

    // 3. Add timestamp context
    validation.sent_at = device_get_timestamp();

    // 4. Convert to device_response_t (Layer 2)
    // All fields (status, error, payload) copied as-is
    device_response_t response = DeviceResponse::from_event(validation);
    // response now contains all fields from validation

    // 5. Layer 3: Serialize to JSON and send (NO modifications needed)
    // send() serializes all populated fields
    DeviceResponse::send(response);
  }

  // ... rest of loop ...
}

Key Design Principle:

  • Layer 1 (event_to_response): Validates sensor data and populates event_response_t with status/error AND payload (hit1, hit2, hit3, adc, etc.)
  • Layer 2 (from_event): Copies all fields to device_response_t (pure data transfer)
  • Layer 3 (send): Serializes fully-populated device_response_t as-is (no conditional logic)

Error Handling Patterns (Layer 1)

Command Handler Error Patterns

Pattern 1: Argument validation failure

command_response_t handle_set_threshold(const command_t& cmd) {
  // Validate argument count
  if (cmd.arg_count != 2) {
    return device_response_error(DEVICE_TYPE_RESPONSE,
                                 DEVICE_CODE_INVALID_ARG,
                                 "SET_THRESHOLD requires 2 arguments: channel value");
  }

  // Parse arguments
  int channel = strtol(cmd.args[0], nullptr, 10);
  int value = strtol(cmd.args[1], nullptr, 10);

  // Validate ranges
  if (channel < 1 || channel > 3) {
    return device_response_error(DEVICE_TYPE_RESPONSE,
                                 DEVICE_CODE_OUT_OF_RANGE,
                                 "Channel must be 1-3");
  }

  if (value < 0 || value > 1023) {
    return device_response_error(DEVICE_TYPE_RESPONSE,
                                 DEVICE_CODE_OUT_OF_RANGE,
                                 "Threshold must be 0-1023");
  }

  // Success: handler will populate is_ok=true, status=OK
  command_response_t resp = device_response_ok(DEVICE_TYPE_RESPONSE);
  resp.is_ok = true;
  resp.error_code = DEVICE_CODE_OK;
  return resp;
}

Pattern 2: Device not ready

command_response_t handle_get_bme280(const command_t& cmd) {
  // Check if BME280 is initialized
  if (!bme280_is_initialized()) {
    return device_response_error(DEVICE_TYPE_RESPONSE,
                                 DEVICE_CODE_NOT_READY,
                                 "BME280 sensor not initialized");
  }

  // Success: return OK
  command_response_t resp = device_response_ok(DEVICE_TYPE_RESPONSE);
  resp.is_ok = true;
  resp.error_code = DEVICE_CODE_OK;
  return resp;
}

Pattern 3: Unknown command

// In CommandQueue::dispatch()
command_handler_t handler = find_handler(cmd.name);
if (!handler) {
  return device_response_error(DEVICE_TYPE_RESPONSE,
                               DEVICE_CODE_UNKNOWN,
                               "Unknown command: " + std::string(cmd.name));
}

Event Validation Error Patterns

Pattern 1: No active channels

event_response_t event_to_response(const event_t* event) {
  // Validate sensor data
  if (!event || (event->hit1 == 0 && event->hit2 == 0 && event->hit3 == 0)) {
    return device_response_error(DEVICE_TYPE_EVENT,
                                 DEVICE_CODE_NOT_READY,
                                 "No active detection channels");
  }

  // Success: return OK
  event_response_t resp = device_response_ok(DEVICE_TYPE_EVENT);
  resp.is_ok = true;
  resp.error_code = DEVICE_CODE_OK;
  return resp;
}

Pattern 2: Sensor timeout

event_response_t event_to_response(const event_t* event) {
  // Check if event is stale
  uint32_t now = millis();
  if ((now - event->capture_time_ms) > 1000) {  // Older than 1 second
    return device_response_error(DEVICE_TYPE_EVENT,
                                 DEVICE_CODE_TIMEOUT,
                                 "Event data too old (>1000ms)");
  }

  // Success
  event_response_t resp = device_response_ok(DEVICE_TYPE_EVENT);
  resp.is_ok = true;
  resp.error_code = DEVICE_CODE_OK;
  return resp;
}

JSON Output Examples (Layer 3)

Command Error Response

{
  "type": "response",
  "status": "error",
  "sent_at": 12345,
  "error_code": 2,
  "error_message": "Value 999 out of range [0-1023]"
}

When to send:

  • Handler validation failed
  • Argument parsing failed
  • Value out of acceptable range

Event Error Response

{
  "type": "event",
  "status": "error",
  "sent_at": 12345,
  "error_code": 3,
  "error_message": "Sensors not ready: no channels active"
}

When to send:

  • Detection triggered but no valid signal
  • Sensor hardware not initialized
  • Data validation failed

Key Field Behavior

Field Always Present? When status="ok" When status="error"
type ✅ Always RESPONSE or EVENT RESPONSE or EVENT
status ✅ Always "ok" "error"
sent_at ✅ Always Timestamp Timestamp
error_code ✅ Always 0 (DEVICE_CODE_OK) 1-5 (error code)
error_message ✅ Always Empty string Human-readable error
Payload fields ❌ Conditional Present (version, hit1, etc.) NOT present

Important: When status="error", NO payload fields are serialized. Only envelope fields are sent.

Design Benefits

1. Clean Layer Separation

Layer Responsibility Code Location Type
1 Handler logic + validation src/command/*.cpp or main.cpp command_response_t / event_response_t
2 Type conversion DeviceResponse::from_*() device_response_t
3 JSON serialization DeviceResponse::send*() JSON string

2. Handler Simplicity

Handlers focus ONLY on business logic:

  • Parse and validate arguments
  • Execute command logic or validate sensors
  • Return status/error information

Handlers do NOT:

  • Touch JSON construction
  • Know about serialization
  • Deal with timestamps or transport details

3. Type Safety

  • Compile-time validation of structure membership
  • No error-prone string manipulation
  • ArduinoJson provides schema validation

4. Testability

Each layer can be tested independently:

  • Layer 1: Test handler logic without JSON knowledge
  • Layer 2: Test conversion as pure data transformation
  • Layer 3: Test serialization independently

5. Symmetry

Commands and events follow identical pipeline:

Command:  command_t → command_response_t → device_response_t → JSON
Event:    event_t   → event_response_t   → device_response_t → JSON

Both use same DeviceResponse serialization (one class, two use cases).

6. Future Extensibility

  • New payload fields: Add to handler → automatically included
  • New serialization formats: Add send_msgpack(), send_cbor() methods
  • New event types: Define new *_response_t, use same DeviceResponse class

Implementation Checklist

Prerequisites (Phase 0 - Already Done)

  • ✅ Define command_response_t (Layer 1 type for commands)
  • ✅ Define event_response_t (Layer 1 type for events)
  • ✅ Implement builder functions: device_response_ok(), device_response_error()
  • ✅ Implement device_get_timestamp()

Phase 0.5 Implementation

  • Add DeviceResponse class declaration to include/device_response.h
  • Implement from_command() conversion
  • Implement from_event() conversion
  • Implement send() for command serialization
  • Implement send_event() for event serialization
  • Update CommandQueue::dispatch() to use DeviceResponse
  • Update detection loop in main.cpp to use DeviceResponse
  • Test with task monitor (manual serial commands)
  • Verify JSON output against docs/schemas/device-response.json

Success Criteria

  • ✅ All command responses serialized via DeviceResponse::send()
  • ✅ All event responses serialized via DeviceResponse::send_event()
  • ✅ JSON output conforms to device-response.json schema
  • ✅ No manual snprintf() JSON construction in handlers
  • ✅ No changes to handler signatures (still return command_response_t)
  • ✅ Zero impact on handlers (only dispatcher code changes)

Compilation Control

When ENABLE_DEVICE_RESPONSE=1:

  • DeviceResponse class compiled
  • All three layers active
  • device_response_t envelope used

When ENABLE_DEVICE_RESPONSE=0:

  • Legacy response handling (completely separate code path)
  • DeviceResponse class not compiled (zero overhead)
  • Backward compatibility maintained

No overlap between paths. One compile flag, one implementation.

Timeline

  • Implementation: 1-2 days (class + integration)
  • Testing: 1 day (manual verification via task monitor)
  • Total: 2-3 days for Phase 0.5

Next Phase: Phase 1 (Handler Refactoring)

After Phase 0.5 DeviceResponse is complete, Phase 1 will:

  1. Update all 16 command handlers to return command_response_t with structured data (not JSON strings)
  2. Create reusable handler utilities in command/handlers_common.h
  3. Implement command-specific payload building in handlers

Example (Phase 1 implementation in handler):

// BEFORE (current, Phase 0):
command_response_t handle_version(const command_t& cmd) {
  command_response_t resp;
  resp.is_ok = true;
  snprintf(resp.message, sizeof(resp.message),
    "{\"type\":\"response\",\"status\":\"ok\",\"version\":\"%s\"}",
    Command::getInstance().get_version());
  return resp;
}

// AFTER (Phase 1):
command_response_t handle_version(const command_t& cmd) {
  // Handler just returns status/error info
  command_response_t resp = device_response_ok(DEVICE_TYPE_RESPONSE);
  resp.is_ok = true;
  resp.error_code = DEVICE_CODE_OK;
  return resp;
  // Dispatcher will add payload and serialize via DeviceResponse
}

// In CommandQueue::dispatch() (after handler):
command_response_t result = handle_version(cmd);
device_response_t envelope = DeviceResponse::from_command(result);
StaticJsonDocument<256> payload;
payload["version"] = Command::getInstance().get_version();  // Handler-specific payload
DeviceResponse::send(envelope, &payload);

Summary

Phase 0.5 goal: Implement the DeviceResponse class as the unified Layer 2→3 pipeline for all device output (commands and events).

Key design decisions:

  1. Static utility class (no instantiation, all methods static)
  2. Two conversion methods: from_command() and from_event()
  3. Single serialization method: send() (universal for commands and events)
  4. Payload assembly responsibility in caller (CommandQueue or main.cpp)
  5. Layer 3 contains only serialization logic - no domain-specific conditional code
  6. Complete separation from legacy response handling (when ENABLE_DEVICE_RESPONSE=0)

Outcome: Commands and events use identical, type-safe, auditable serialization pipeline. Handlers focus on business logic only. JSON construction centralized in one class.

Current Implementation Reality vs Phase 0.5 Design

Current Pattern: DeviceResponseBuilder in text_command_manager.cpp

Status: Already implemented and actively used (when ENABLE_DEVICE_RESPONSE=1)

Current usage (src/text_command_manager.cpp:250-263):

// Layer 1: Handler directly builds and outputs JSON
if (config_set_poll_count(count)) {
#if ENABLE_DEVICE_RESPONSE
    // Direct JSON construction in handler
    JsonDocument doc = DeviceResponseBuilder::simple("poll_count", (int32_t)config_get_poll_count());
    serializeJson(doc, Serial);
    Serial.println();
    response.is_ok = true;
#else
    // Legacy behavior
    send_response(response_int("poll_count", config_get_poll_count()));
#endif
}

Pattern Analysis:

  • JSON construction happens in Layer 1 (handler)
  • No intermediate data type conversion (skips Layer 2)
  • Direct Serial output from handler (no Layer 3 separation)
  • DeviceResponseBuilder provides reusable JSON envelope construction

Design Gap: Two Competing Patterns

Aspect Current (text_command_manager) Phase 0.5 Design Status
JSON Generation Layer 1 (handler) Layer 3 (send) ❌ Conflict
Data Type JsonDocument device_response_t struct ❌ Conflict
Separation Handler handles JSON Dispatcher handles Layer 2→3 ❌ Conflict
Builder Usage Direct in handler Delegated from send() 🔄 Unclear

Based on analysis of:

  1. Current implementation (text_command_manager uses builder in Layer 1)
  2. Phase 0.5 design (unified Layer 2→3 conversion)
  3. Code reusability and maintainability

Recommendation: Layer 3 (DeviceResponse::send) should own the builder pattern

Rationale

Layer 1 Focus: Handler/Validator should return typed data structures only:

  • command_response_t with status/error fields (no JSON)
  • event_response_t with status/error fields (no JSON)
  • Handlers focus purely on business logic and validation

Layer 2 Purpose: Pure data type conversion with no side effects:

  • DeviceResponse::from_command(command_response_t) → device_response_t
  • DeviceResponse::from_event(event_response_t) → device_response_t
  • Copy all fields from Layer 1 → Layer 2 struct
  • No JSON construction

Layer 3 Authority: Unified JSON construction and serialization:

  • DeviceResponse::send(device_response_t) orchestrates serialization
  • Internally uses helper methods for JSON building (inherited from DeviceResponseBuilder pattern)
  • Handles all envelope fields (type, status, sent_at, error fields)
  • Handles conditional error_code/error_message serialization
  • Single point of JSON output logic for entire system

Implementation Strategy

Option A: Integrate DeviceResponseBuilder → DeviceResponse

Move builder methods into DeviceResponse class as private helpers:

class DeviceResponse {
 public:
  // Layer 2→3
  static device_response_t from_command(const command_response_t& response);
  static device_response_t from_event(const event_response_t& response);
  static void send(const device_response_t& response);

 private:
  // Layer 3 JSON helpers (derived from DeviceResponseBuilder pattern)
  static void init_common_fields(JsonDocument& doc);
  static void add_field(JsonDocument& doc, const char* name, const char* value);
  static void add_field(JsonDocument& doc, const char* name, int32_t value);
  // ... etc
};

Advantages:

  • ✅ Single class owns Layer 2→3 responsibility
  • ✅ DeviceResponseBuilder can be retired (no longer needed)
  • ✅ Unified interface for all output

Disadvantages:

  • ❌ DeviceResponse grows larger
  • ❌ Private helpers less reusable
Option B: Keep DeviceResponseBuilder as Layer 3 helper

Keep builder pattern intact, have DeviceResponse::send() delegate to it:

class DeviceResponse {
 public:
  static device_response_t from_command(const command_response_t& response);
  static device_response_t from_event(const event_response_t& response);
  static void send(const device_response_t& response);
  // Handlers NOT allowed to use builder directly
};

// DeviceResponseBuilder remains Layer 3 helper (internal to send())
class DeviceResponseBuilder {
 private:  // ← Changed from public
  static JsonDocument simple(...);
  static JsonDocument empty();
  // Only accessible from DeviceResponse::send()
  friend class DeviceResponse;
};

Advantages:

  • ✅ Existing builder pattern preserved
  • ✅ Clean separation of concerns
  • ✅ Reusable builder logic

Disadvantages:

  • ❌ Extra class indirection
  • ❌ More complex to understand flow

Transition Strategy: Phase 0.5 vs Current

Phase 0.5a: Reevaluate Current Implementation

CURRENT STATE (text_command_manager):
  handler() → [is_ok, message_json] → Serial.println(message)

PHASE 0.5 IDEAL (CommandQueue):
  handler() → [command_response_t] → from_command() → send()

Decision Point:

  • Keep text_command_manager pattern (simpler for legacy)
  • Require CommandQueue to follow Phase 0.5 pattern (with builder)

Phase 0.5b: CommandQueue with Unified Builder

When CommandQueue handlers are implemented:

// Layer 1: Handler returns typed data
command_response_t handle_version(const command_t& cmd) {
  auto resp = device_response_ok(DEVICE_TYPE_RESPONSE);
  resp.version = "v1.14.0";  // No JSON
  resp.uptime_ms = millis();
  return resp;
}

// Layer 2→3: DeviceResponse handles everything
device_response_t response = DeviceResponse::from_command(handler_result);
DeviceResponse::send(response);  // Internal: Uses builder pattern

JSON Builder Pattern: Why Layer 3?

Principle: JSON is a transmission format, not a data structure

  1. Layer 1: Work with native C++ types (command_response_t)
  2. Layer 2: Convert between type systems (device_response_t)
  3. Layer 3: Format for transmission (JSON serialization)

Layering Benefits

┌─────────────────────────────────────────┐
│ BEFORE (Handler Creates JSON)           │
├─────────────────────────────────────────┤
│ Handlers scattered:                     │
│  - Some use DeviceResponseBuilder       │
│  - Some use manual serializeJson()      │
│  - Some use different patterns          │
│ Problem: Multiple serialization paths   │
│ Problem: Hard to audit JSON correctness │
└─────────────────────────────────────────┘

┌─────────────────────────────────────────┐
│ AFTER (DeviceResponse::send Owns JSON)  │
├─────────────────────────────────────────┤
│ All JSON routes through:                │
│  - DeviceResponse::send() (command)     │
│  - DeviceResponse::send() (event)       │
│ Benefit: Single audit point             │
│ Benefit: Consistent output              │
│ Benefit: Schema validation in one place │
│ Benefit: Easy to add logging/debugging  │
└─────────────────────────────────────────┘

Recommendation Summary

Component Responsibility Location Pattern
Layer 1 Business logic + validation Handlers Return command_response_t / event_response_t (no JSON)
Layer 2 Type conversion DeviceResponse::from_*() Pure data copy (no JSON)
Layer 3 JSON serialization DeviceResponse::send() Use builder pattern internally
Builder JSON construction helpers Private to DeviceResponse Not exposed to handlers

For CommandQueue Phase 1:

  • Handlers return typed data only
  • DeviceResponse owns all JSON generation
  • Builder pattern is an implementation detail of send()

For text_command_manager (legacy):

  • Continue current pattern (lower priority for refactoring)
  • Eventually migrate to Phase 0.5 pattern
  • DeviceResponseBuilder can remain public for backward compatibility

Command Handler Migration: Detailed Refactoring Guide

Current Implementation Status

All command handlers in src/command/*.cpp currently follow Layer 1 + Direct JSON pattern:

  1. Handler receives command_t (parsed arguments)
  2. Handler executes logic and validates arguments
  3. Handler builds JSON using snprintf() directly
  4. JSON stored in command_response_t.message[512]
  5. Caller outputs response.message via Serial

Problem: JSON generation is mixed into handler logic (Layer 1 responsibility)

Phase 0.5 Target Pattern

Handlers should follow Layer 1 Only pattern:

  1. Handler receives command_t (parsed arguments)
  2. Handler executes logic and validates arguments
  3. Handler populates typed fields in command_response_t
  4. Handler returns command_response_t (status/error + payload fields only)
  5. Dispatcher calls DeviceResponse::from_command() and DeviceResponse::send()

Benefit: Clean separation - handlers don't know about JSON format

Step-by-Step Migration Process

Step 1: Update command_response_t Structure

Location: include/command_queue.h:140-161

Before (Current):

typedef struct {
  bool is_ok;              // Handler execution success
  char message[512];       // JSON response (will be replaced)
} command_response_t;

After (Phase 0.5):

typedef struct {
  // Layer 1 Status/Error fields (handler populates these)
  bool is_ok;              // Handler execution success
  device_response_status_t status;   // OK or ERROR
  device_response_code_t error_code; // Error classification (0-5)
  char error_message[256]; // Human-readable error
  uint32_t sent_at;        // Timestamp from dispatcher
  device_response_type_t type; // DEVICE_TYPE_RESPONSE (always for commands)

  // Layer 1 Payload fields (handler populates command-specific fields)
  // Example for GET_VERSION:
  char version[32];        // Firmware version string

  // Example for GET_STATUS:
  uint32_t uptime_ms;      // Device uptime milliseconds
  char mac_address[18];    // MAC address string (hex format)
  uint16_t poll_count;     // Detection poll count
  uint16_t deadtime_ms;    // Detector deadtime milliseconds

  // Example for GET_THRESHOLD/SET_THRESHOLD:
  uint8_t channel;         // Threshold channel (1-3)
  uint16_t threshold;      // Threshold value (0-1023)

  // ... additional fields for other commands (stream, poll_count, etc.)
} command_response_t;

Key Changes:

  • ❌ Remove: message[512] (JSON field)
  • ✅ Add: status, error_code, error_message (error handling)
  • ✅ Add: sent_at, type (envelope metadata)
  • ✅ Add: Command-specific payload fields (version, uptime_ms, channel, threshold, etc.)

Step 2: Implement DeviceResponse Class

Location: src/device_response.cpp and include/device_response.h

Add implementation of Layer 2→3 conversion methods (currently only declarations exist):

// include/device_response.h - Add to class declaration:

class DeviceResponse {
 public:
  // ... existing declarations ...

  /**
   * @brief Convert command handler result → device response
   *
   * Layer 1→2 conversion: command_response_t → device_response_t
   * Copies all status/error fields AND payload fields from handler result.
   *
   * @param response Command handler result (Layer 1)
   * @return device_response_t Fully populated, ready for serialization (Layer 2)
   */
  static device_response_t from_command(const command_response_t& response);

  /**
   * @brief Serialize device response to JSON Lines on Serial
   *
   * Layer 3 serialization: device_response_t → JSON → Serial output
   * Converts all response fields to JSON Lines format (single line per message).
   *
   * @param response Fully populated device response (Layer 2)
   * @return void (outputs to Serial)
   *
   * Serialization rules:
   * - Always: type, status, sent_at
   * - When status=error: error_code, error_message
   * - When status=ok: all populated payload fields
   */
  static void send(const device_response_t& response);
};

Implementation in src/device_response.cpp:

device_response_t DeviceResponse::from_command(const command_response_t& response) {
  device_response_t result = {};

  // Copy envelope fields
  result.type = DEVICE_TYPE_RESPONSE;
  result.status = response.status;
  result.error_code = response.error_code;
  result.sent_at = response.sent_at;

  // Copy error message
  strncpy(result.error_message, response.error_message,
          sizeof(result.error_message) - 1);
  result.error_message[sizeof(result.error_message) - 1] = '\0';

  // Copy payload fields (all fields from command_response_t)
  if (response.version[0]) {
    strncpy(result.version, response.version, sizeof(result.version) - 1);
  }
  if (response.mac_address[0]) {
    strncpy(result.mac_address, response.mac_address, sizeof(result.mac_address) - 1);
  }
  result.uptime_ms = response.uptime_ms;
  result.poll_count = response.poll_count;
  result.deadtime_ms = response.deadtime_ms;
  result.channel = response.channel;
  result.threshold = response.threshold;
  // ... copy other payload fields ...

  return result;
}

void DeviceResponse::send(const device_response_t& response) {
  StaticJsonDocument<512> doc;

  // 1. Envelope fields (always)
  doc["type"] = (response.type == DEVICE_TYPE_RESPONSE) ? "response" : "event";
  doc["status"] = (response.status == DEVICE_STATUS_OK) ? "ok" : "error";
  doc["sent_at"] = response.sent_at;

  // 2. Error fields (only when error)
  if (response.status == DEVICE_STATUS_ERROR) {
    doc["error_code"] = response.error_code;
    doc["error_message"] = response.error_message;
  }

  // 3. Payload fields (only when status=ok)
  if (response.status == DEVICE_STATUS_OK) {
    if (response.version[0]) {
      doc["version"] = response.version;
    }
    if (response.mac_address[0]) {
      doc["mac_address"] = response.mac_address;
    }
    if (response.uptime_ms > 0) {
      doc["uptime_ms"] = response.uptime_ms;
    }
    if (response.poll_count > 0) {
      doc["poll_count"] = response.poll_count;
    }
    if (response.deadtime_ms > 0) {
      doc["deadtime_ms"] = response.deadtime_ms;
    }
    if (response.channel > 0) {
      doc["channel"] = response.channel;
      doc["threshold"] = response.threshold;
    }
    // ... serialize other payload fields ...
  }

  serializeJson(doc, Serial);
  Serial.println();
}

Step 3: Create Error Helper Functions

Location: src/command_manager.cpp or include/command_manager.h

Add helpers for creating error responses (currently used as error_response() in handlers):

// Helper function for command handlers to create error responses
inline command_response_t error_response(uint16_t code, const char* message) {
  command_response_t resp = {};
  resp.is_ok = false;
  resp.status = DEVICE_STATUS_ERROR;
  resp.error_code = (device_response_code_t)code;
  resp.type = DEVICE_TYPE_RESPONSE;
  strncpy(resp.error_message, message, sizeof(resp.error_message) - 1);
  return resp;
}

// Helper function for successful responses
inline command_response_t success_response() {
  command_response_t resp = {};
  resp.is_ok = true;
  resp.status = DEVICE_STATUS_OK;
  resp.error_code = DEVICE_CODE_OK;
  resp.type = DEVICE_TYPE_RESPONSE;
  return resp;
}

Step 4: Update CommandQueue::execute()

Location: src/command_queue.cpp

Modify dispatcher to call DeviceResponse layer:

Before:

static void execute() {
  command_t cmd = {};
  if (dequeue(&cmd)) {
    command_response_t response = dispatch(&cmd);
    Serial.println(response.message);  // ← Direct Serial output
  }
}

After:

static void execute() {
  command_t cmd = {};
  if (dequeue(&cmd)) {
    // Layer 1: Dispatch to handler
    command_response_t handler_result = dispatch(&cmd);

    // Add dispatcher context
    handler_result.sent_at = device_get_timestamp();

    // Layer 2: Convert to device_response_t
    device_response_t response = DeviceResponse::from_command(handler_result);

    // Layer 3: Serialize and send (no modifications needed)
    DeviceResponse::send(response);
  }
}

Concrete Handler Examples

Example 1: GET_VERSION Command

Before (Current - src/command/version.cpp):

command_response_t handle_get_version(const command_t& cmd) {
  // Validate no arguments provided
  if (cmd.arg_count != 0) {
    return error_response(1, "GET_VERSION takes no arguments");
  }

  // Get version from Command class singleton
  const char* version = Command::getInstance().get_version();

  // Create success response with version field
  command_response_t response;
  response.is_ok = true;
  snprintf(response.message, sizeof(response.message),
    "{\"type\":\"response\",\"status\":\"ok\",\"version\":\"%s\"}",
    version);  // ← JSON built in handler (Layer 1)

  return response;
}

After (Phase 0.5):

command_response_t handle_get_version(const command_t& cmd) {
  // Validate no arguments provided
  if (cmd.arg_count != 0) {
    return error_response(DEVICE_CODE_INVALID_ARG, "GET_VERSION takes no arguments");
  }

  // Get version from Command class singleton
  const char* version = Command::getInstance().get_version();

  // Create success response (no JSON!)
  command_response_t response = success_response();

  // Populate version field (handler responsibility)
  strncpy(response.version, version, sizeof(response.version) - 1);
  response.version[sizeof(response.version) - 1] = '\0';

  return response;
  // JSON will be built by DeviceResponse::send() in Layer 3
}

Output from both versions:

{"type":"response","status":"ok","sent_at":12345,"version":"v1.14.0"}

Key Differences:

  • ❌ Before: JSON constructed with snprintf() in handler
  • ✅ After: Only C++ string copied to struct field
  • ❌ Before: Handler knows JSON format
  • ✅ After: Handler only knows domain logic
  • ✅ After: JSON construction delegated to Layer 3

Example 2: SET_THRESHOLD Command

Before (Current - src/command/threshold.cpp):

command_response_t handle_set_threshold(const command_t& cmd) {
  if (cmd.arg_count != 2) {
    return error_response(1, "SET_THRESHOLD requires two arguments: <ch> <val>");
  }

  char* endptr;
  uint8_t ch = (uint8_t)strtoul(cmd.args[0], &endptr, 10);
  if (*endptr != '\0' || ch < 1 || ch > 3) {
    return error_response(2, "Invalid channel (must be 1-3)");
  }

  uint16_t val = (uint16_t)strtoul(cmd.args[1], &endptr, 10);
  if (*endptr != '\0' || val > 1023) {
    return error_response(2, "Invalid threshold (must be 0-1023)");
  }

  // Set threshold via Command class (includes DAC sync)
  Command::getInstance().set_threshold(ch, val);

  command_response_t response;
  response.is_ok = true;
  snprintf(response.message, sizeof(response.message),
    "{\"type\":\"response\",\"status\":\"ok\",\"channel\":%d,\"threshold\":%u}",
    ch, val);  // ← JSON built in handler
  return response;
}

After (Phase 0.5):

command_response_t handle_set_threshold(const command_t& cmd) {
  if (cmd.arg_count != 2) {
    return error_response(DEVICE_CODE_INVALID_ARG,
                         "SET_THRESHOLD requires two arguments: <ch> <val>");
  }

  char* endptr;
  uint8_t ch = (uint8_t)strtoul(cmd.args[0], &endptr, 10);
  if (*endptr != '\0' || ch < 1 || ch > 3) {
    return error_response(DEVICE_CODE_OUT_OF_RANGE,
                         "Invalid channel (must be 1-3)");
  }

  uint16_t val = (uint16_t)strtoul(cmd.args[1], &endptr, 10);
  if (*endptr != '\0' || val > 1023) {
    return error_response(DEVICE_CODE_OUT_OF_RANGE,
                         "Invalid threshold (must be 0-1023)");
  }

  // Set threshold via Command class (includes DAC sync)
  Command::getInstance().set_threshold(ch, val);

  // Create success response (no JSON!)
  command_response_t response = success_response();

  // Populate threshold fields (handler responsibility)
  response.channel = ch;
  response.threshold = val;

  return response;
  // JSON will be built by DeviceResponse::send() in Layer 3
}

Output from both versions:

{"type":"response","status":"ok","sent_at":12345,"channel":1,"threshold":512}

Error output example:

{"type":"response","status":"error","sent_at":12345,"error_code":2,"error_message":"Invalid threshold (must be 0-1023)"}

Key Differences:

  • ❌ Before: Hard-coded error codes (1, 2) - unclear meaning
  • ✅ After: Semantic error codes (DEVICE_CODE_INVALID_ARG, DEVICE_CODE_OUT_OF_RANGE)
  • ❌ Before: JSON format must match schema manually
  • ✅ After: JSON generation guaranteed by DeviceResponse::send()
  • ✅ After: Error messages stored in structured field, not embedded in JSON string

Example 3: GET_THRESHOLD Command (Read-Only)

Before (Current):

command_response_t handle_get_threshold(const command_t& cmd) {
  if (cmd.arg_count != 1) {
    return error_response(1, "GET_THRESHOLD requires one argument: <ch>");
  }

  char* endptr;
  uint8_t ch = (uint8_t)strtoul(cmd.args[0], &endptr, 10);
  if (*endptr != '\0' || ch < 1 || ch > 3) {
    return error_response(2, "Invalid channel (must be 1-3)");
  }

  uint16_t val = Command::getInstance().get_threshold(ch);

  command_response_t response;
  response.is_ok = true;
  snprintf(response.message, sizeof(response.message),
    "{\"type\":\"response\",\"status\":\"ok\",\"channel\":%d,\"threshold\":%u}",
    ch, val);  // ← JSON built in handler
  return response;
}

After (Phase 0.5):

command_response_t handle_get_threshold(const command_t& cmd) {
  if (cmd.arg_count != 1) {
    return error_response(DEVICE_CODE_INVALID_ARG,
                         "GET_THRESHOLD requires one argument: <ch>");
  }

  char* endptr;
  uint8_t ch = (uint8_t)strtoul(cmd.args[0], &endptr, 10);
  if (*endptr != '\0' || ch < 1 || ch > 3) {
    return error_response(DEVICE_CODE_OUT_OF_RANGE,
                         "Invalid channel (must be 1-3)");
  }

  uint16_t val = Command::getInstance().get_threshold(ch);

  // Create success response (no JSON!)
  command_response_t response = success_response();

  // Populate threshold fields
  response.channel = ch;
  response.threshold = val;

  return response;
  // JSON will be built by DeviceResponse::send() in Layer 3
}

Pattern: Identical to SET_THRESHOLD handler structure (error handling + field population)

Migration Checklist

Apply to all 15 command handlers in src/command/*.cpp:

  • version.cpp: handle_get_version()
  • Fields: version

  • Errors: INVALID_ARG (no arguments expected)

  • status.cpp: handle_get_status()

  • Fields: uptime_ms, mac_address, poll_count, deadtime_ms

  • Errors: INVALID_ARG

  • uptime.cpp: handle_get_uptime()

  • Fields: uptime_ms

  • Errors: INVALID_ARG

  • mac_address.cpp: handle_get_mac_address()

  • Fields: mac_address

  • Errors: INVALID_ARG

  • threshold.cpp: handle_set_threshold(), handle_get_threshold()

  • Fields: channel, threshold

  • Errors: INVALID_ARG, OUT_OF_RANGE

  • poll_count.cpp: handle_set_poll_count(), handle_get_poll_count()

  • Fields: poll_count

  • Errors: INVALID_ARG, OUT_OF_RANGE

  • deadtime.cpp: handle_set_deadtime(), handle_get_deadtime()

  • Fields: deadtime_ms

  • Errors: INVALID_ARG, OUT_OF_RANGE

  • stream.cpp: handle_set_stream(), handle_get_stream()

  • Fields: enabled (boolean as uint8_t)

  • Errors: INVALID_ARG

  • rtc.cpp: handle_set_rtc_time(), handle_get_rtc_time()

  • Fields: unix_timestamp

  • Errors: INVALID_ARG, NOT_READY

  • bme280.cpp: handle_get_bme280()

  • Fields: temperature, pressure, humidity

  • Errors: NOT_READY (sensor not initialized)

  • gnss.cpp: GNSS command handlers

  • Fields: latitude, longitude, altitude, satellites, fix_quality

  • Errors: NOT_READY

  • wifi.cpp: WiFi command handlers

  • Fields: ssid, rssi, status

  • Errors: INVALID_ARG, NOT_READY

  • test_led.cpp: handle_test_led()

  • Fields: None (empty success response)

  • Errors: INVALID_ARG, OUT_OF_RANGE

  • reset.cpp: handle_reset()

  • Fields: None (no response before reset)

  • Errors: None

  • help.cpp: handle_help()

  • Fields: help_text (or structured command list)
  • Errors: INVALID_ARG

Validation & Testing

After refactoring each handler:

  1. Compile check:
task build
  1. Serial output check (via task monitor):
{"type":"response","status":"ok","sent_at":12345,"version":"v1.14.0"}
  1. Error case check:
{"type":"response","status":"error","sent_at":12345,"error_code":2,"error_message":"Invalid threshold (must be 0-1023)"}
  1. Schema validation (optional):
# Validate against strict schema
cat output.json | jq '.' | validate-json-schema docs/schemas/device-response.json

Summary of Changes

Aspect Before After
JSON Location Handler (Layer 1) DeviceResponse::send() (Layer 3)
Response Type command_response_t.message[512] (JSON string) command_response_t (typed struct with payload fields)
Error Handling Hard-coded numbers (1, 2) Semantic codes (DEVICE_CODE_INVALID_ARG, etc.)
Payload Assembly In handler using snprintf Direct struct field assignment
Serialization Handler responsibility Dispatcher responsibility (DeviceResponse class)
Testability Handler output is JSON text Handler output is typed struct (easier unit testing)
Schema Compliance Manual verification Automatic by DeviceResponse::send()
Code Duplication High (every handler has JSON format logic) Zero (all JSON centralized)

Implementation Status: Phase 6 Payload Pointer Pattern (v1.13.12)

Evolution from Phase 0.5 Design to Phase 6 Reality

This design document outlines Phase 0.5 (published 2025-12-10), which proposed:

  • command_response_t with typed payload fields (version, uptime_ms, mac_address, etc.)
  • DeviceResponse class with from_command() → device_response_t conversion
  • Serialization via DeviceResponse::send() with field-by-field conditional logic

However, the actual implementation (Phase 6, v1.13.12, 2025-12-11) evolved into a more efficient pattern that maintains the same layer separation but eliminates the need for large typed structs.

Phase 6: Actual Implementation Pattern

Instead of embedding typed fields in command_response_t, Phase 6 uses pointer-based payloads:

command_response_t (Phase 6 Reality):

typedef struct {
  const JsonDocument* payload;  // Points to handler's static JsonDocument
} command_response_t;

device_response_t (Phase 6 Reality):

typedef struct {
  const JsonDocument* payload;  // Merged with envelope by Layer 3
} device_response_t;

Why Phase 6 > Phase 0.5 Design

Aspect Phase 0.5 Design Phase 6 Reality Benefit
Structure Size ~230 bytes (all fields) 8 bytes (pointer only) 96.5% memory reduction
Payload Flexibility Predefined typed fields Any JSON structure No struct changes for new commands
Handler Pattern Assign to struct fields Create static JsonDocument Cleaner separation of concerns
Serialization Conditional per field Merge JsonDocument Unified algorithm, no hardcoding
Type Safety Compile-time struct validation Runtime ArduinoJson validation Equivalent safety, more flexibility

Key Design Principle Maintained

Both Phase 0.5 design and Phase 6 implementation maintain the same three-layer architecture:

Layer 1 (Handler):
  └─> Receive command_t, execute logic, validate arguments
  └─> Return command_response_t with payload (response_t or JsonDocument*)

    ↓ (Dispatcher passes through)

Layer 2 (Conversion):
  └─> Copy command_response_t → device_response_t
  └─> No transformation needed (payload is already prepared)

    ↓ (Serializer merges)

Layer 3 (Serialization):
  └─> Build JSON envelope (type, status, sent_at, error fields)
  └─> Merge payload (Phase 0.5: add fields conditionally; Phase 6: iterate JsonDocument)
  └─> Output JSON Lines to Serial

Handler Code Comparison

Phase 0.5 Design (what was planned):

command_response_t handle_get_version(const command_t& cmd) {
  command_response_t resp = success_response();
  strncpy(resp.version, Command::getInstance().get_version(), sizeof(resp.version) - 1);
  return resp;
}

Phase 6 Implementation (what was actually built):

command_response_t handle_get_version(const command_t& cmd) {
  static JsonDocument payload;
  payload.clear();
  payload["version"] = Command::getInstance().get_version();

  command_response_t response = success_response();
  response.payload = &payload;  // Point to JsonDocument
  return response;
}

Outcome: Both produce identical JSON output, Phase 6 is more flexible and memory-efficient.

References to Phase 6 Implementation

For detailed Phase 6 architecture and implementation, see:

Conclusion: Design vs Reality

This Phase 0.5 design document remains valuable as architectural reference showing:

  • Clean layer separation principles (maintained in Phase 6)
  • Error handling patterns (unchanged)
  • JSON schema compliance (maintained)
  • Type-safe conversion methodology (evolved into JsonDocument approach)

The Phase 6 implementation is the production realization of these principles, optimized for:

  • Memory efficiency (96.5% reduction)
  • Handler flexibility (no struct field limits)
  • Type safety through ArduinoJson library
  • Maintainability (single JSON merging algorithm)