Skip to content

Chapter 4: Middleware Protocol Services & HAL

In Chapter 3, we explored how AIPC lets components talk to each other across process boundaries. Now let's see what lives on the other side of those IPC calls — the middleware protocol services that actually manage your Zigbee, Z-Wave, and WiFi radios.

The Layered Architecture

When the Hub SDK wants to, say, turn on a Z-Wave light bulb, the request doesn't go straight to the radio. It passes through a carefully layered stack:

Hub SDK  →  AIPC  →  Middleware Service  →  HAL  →  Protocol Gateway  →  Radio

Each layer has a single job. The middleware service manages protocol logic and state. The HAL abstracts hardware differences. The protocol gateway speaks the actual radio protocol. This separation means you can swap out hardware without rewriting business logic.

Z-Wave Middleware: A Concrete Example

The Z-Wave middleware service is the best place to start. It runs as a daemon, manages the Z-Wave network's state machine, and exposes a client-facing API through AIPC.

Initialization Lifecycle

Before anything works, a client must initialize the manager and wait for the service to be ready:

// File: iotmi-ace-zwave-mw/include/ace/zwave_mgr.h
aceZWaveMgr_init();          // register with AIPC
if (!aceZWaveMgr_isReady())  // service may still be starting
    // subscribe to ACE_ZWAVE_MGR_NETWORK_READY

This init → isReady → wait-for-ready-event pattern appears in every middleware service. It accounts for the fact that radio hardware takes time to boot.

Network State Machine

The Z-Wave middleware tracks the network through a set of well-defined states:

// File: iotmi-ace-zwave-mw/include/ace/zwave_mgr.h
typedef enum {
    ACE_ZWAVE_NET_STATE_DOWN,
    ACE_ZWAVE_NET_STATE_UP_IDLE,
    ACE_ZWAVE_NET_STATE_UP_IN_NWI,  // inclusion mode
    ACE_ZWAVE_NET_STATE_UP_IN_NWE,  // exclusion mode
    // ...
} aceZWaveMgr_networkState_t;

Only one network operation can run at a time. If you try to start inclusion while an update is running, you'll get ACE_STATUS_BUSY. This is the HSM (Hierarchical State Machine) pattern at work — the service enforces valid transitions.

Event-Driven Callbacks

Async operations like device pairing report results through registered callbacks:

// File: iotmi-ace-zwave-mw/include/ace/zwave_mgr.h
typedef void (*aceZWaveMgr_networkCallback_t)(
    aceZWaveMgr_networkEvent_t event_type,
    const void* data, uint32_t data_len,
    void* client_ctx);

Clients register for specific events using bitmask OR combinations — for example, ACE_ZWAVE_MGR_NETWORK_NODE_ADDED | ACE_ZWAVE_MGR_NETWORK_NODE_REMOVED. This keeps callbacks focused and avoids unnecessary noise.

Sending Commands to Devices

To control a Z-Wave device, the middleware provides command class (CC) APIs. A command targets a specific node and endpoint:

// File: iotmi-ace-zwave-mw/include/ace/zwave_mgr_cc.h
// Send a command class command to a node/endpoint
// Response arrives via aceZWaveMgr_ccReportCallback_t

The response doesn't come back synchronously — it arrives later through a report callback. This two-phase pattern (send → async report) reflects the reality of wireless communication where responses may be delayed or lost.

Zigbee Middleware: Same Pattern, Different Protocol

The Zigbee middleware follows the same architectural pattern. Compare the initialization:

// File: iotmi-ace-general/middleware/zigbee/api/include/ace/zb_adapter.h
aceZb_adapterInit();    // register with AIPC
aceZb_adapterEnable();  // power on radio and stacks

And the event-driven network management:

// File: iotmi-ace-general/middleware/zigbee/api/include/ace/zb_types.h
typedef enum {
    ACE_ZB_NETWORK_FORMED,
    ACE_ZB_NETWORK_DEVICE_JOINED,
    ACE_ZB_NETWORK_DEVICE_DISCOVERED,
    ACE_ZB_NETWORK_DEVICE_LEFT,
    // ...
} aceZb_networkEventType_t;

Sending ZCL cluster commands follows the same async pattern — you get a confirmation_id back immediately, then match it against later events to confirm delivery:

// File: iotmi-ace-general/middleware/zigbee/api/include/ace/zb_command.h
aceZb_clusterSendCmd(dest_node, profile, endpoint,
    src_ep, cluster_id, data, size, &confirm_id);

The consistency across protocols is intentional. Once you understand one middleware service, you understand them all.

The HAL Abstraction Layer

Below each middleware service sits a Hardware Abstraction Layer (HAL). The HAL defines a C interface that the middleware calls, and platform vendors implement.

Z-Wave HAL

The Z-Wave HAL mirrors the middleware API but operates at the hardware level:

// File: iotmi-ace-general/dpk/hal/include/ace/zwave/hal_zwave_core.h
ace_status_t aceZWaveHal_init();
ace_status_t aceZWaveHal_Start();
ace_status_t aceZWaveHal_sendCC(node_id, ep_id,
    cmd_buf, cmd_buf_len);

The HAL reports events back up to the middleware through a registered handler:

// File: iotmi-ace-general/dpk/hal/include/ace/zwave/hal_zwave_core.h
typedef ace_status_t (*aceZWaveHal_eventHandler_t)(
    aceZWaveHal_eventType_t type,
    uint8_t* data, uint32_t dataLen);

Notice the symmetry: the middleware exposes aceZWaveMgr_networkCallback_t to clients, and consumes aceZWaveHal_eventHandler_t from the HAL. Events flow up; commands flow down.

Other HAL Interfaces

The same pattern extends to every hardware component the hub touches:

HAL Header Purpose
hal_zwave_core.h Z-Wave radio operations
hal_wifi_core.h WiFi radio and network stack
hal_device_info.h Serial number, MAC addresses, firmware version
hal_kv_storage.h Persistent key-value storage

For example, hal_device_info.h provides a simple getter interface for device metadata:

// File: iotmi-ace-general/dpk/hal/include/ace/hal_device_info.h
int aceDeviceInfoDsHal_getEntry(
    acehal_device_info_entry_t entry,
    char* data, uint32_t data_size);

Stubs vs. Real Implementations

The SDK ships with two kinds of HAL implementations:

  • Stubs (in dpk/hal/stubs/) — return hardcoded values or no-ops. Used for testing and development without real hardware.
  • Production implementations (in iotmi-ace-dpk/) — talk to actual hardware, often through D-Bus or direct serial interfaces.

This is what makes the HAL so valuable: you can develop and test middleware logic on a laptop using stubs, then swap in real implementations for the target hardware without changing a single line of middleware code.

Protocol Gateways

At the very bottom of the stack sit the protocol gateways — the programs that actually speak to radio hardware:

  • iotmi-ace-zware/ — The Z-Wave gateway (based on the Z/IP framework). It communicates with the Z-Wave controller chip over a serial interface, translating HAL calls into Z-Wave protocol frames (zip_frame.c, zip_transport.c, zip_dtls.c).

  • iotmi-ace-z3-gateway/ — The Zigbee gateway (based on Silicon Labs Gecko SDK v4.3.2). It manages the Zigbee coordinator radio using the EZSP (EmberZNet Serial Protocol) over a host-to-NCP serial link.

You typically don't interact with gateways directly. The HAL layer wraps them, and the middleware wraps the HAL.

End-to-End: A Command's Journey

Here's what happens when the Hub SDK tells a Z-Wave light to turn on:

sequenceDiagram
    participant App as Hub SDK
    participant MW as Z-Wave Middleware
    participant HAL as Z-Wave HAL
    participant GW as Z/IP Gateway
    participant Radio as Z-Wave Radio

    App->>MW: aceZWaveMgr_ccSendCmd(node, ep, switchOn)
    MW->>HAL: aceZWaveHal_sendCC(node, ep, cmd_buf)
    HAL->>GW: Z/IP frame over serial
    GW->>Radio: RF transmission
    Radio-->>GW: ACK from device
    GW-->>HAL: aceZWaveHal_eventHandler(CC_REPORT)
    HAL-->>MW: event propagation
    MW-->>App: aceZWaveMgr_ccReportCallback(result)

The command flows down through four layers, and the response bubbles back up through the same layers via callbacks. Every boundary is a clean interface — you could replace the Z/IP gateway with a different Z-Wave stack and only the HAL implementation would change.

Key Takeaways

  • Middleware services are protocol-specific daemons (Z-Wave, Zigbee, WiFi) that manage network state machines and expose APIs via AIPC.
  • The HAL is a C interface contract between middleware and hardware. Stubs enable testing; production implementations talk to real radios.
  • Protocol gateways (Z/IP for Z-Wave, Gecko SDK for Zigbee) handle the raw radio communication.
  • The pattern is consistent: init → wait-for-ready → send-commands → receive-async-events works the same way across all protocols.
  • Events flow up, commands flow down — this is the fundamental communication model of the entire middleware stack.

What's Next

The middleware services handle local protocol communication, but the hub also needs to talk to the cloud. In Chapter 5: MQTT Proxy, we'll see how the SDK bridges local device events to AWS IoT Core, turning local Z-Wave and Zigbee messages into MQTT publishes — and how cloud commands flow back down to your devices.