Skip to content

Home

Jump to bottom
bkruggelhq edited this page Nov 24, 2025 · 6 revisions

Hilo Connect ‐ Demand Response Integration

Schema

Repository for development of public schemas (repo will remain internal only until a future date)

- -
📝 The schema for demandresponse v1.1 (release 37) is published at https://schema.hiloenergie.com/json/v1_1/demandresponse.schema.json
📝 The schema for demandresponse v1 (release 36) is published at https://schema.hiloenergie.com/json/v1/demandresponse.schema.json
📝 The delivery envelope is defined by the CloudEvents 1.0 spec and is published at https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/formats/cloudevents.json

Messaging

See Telemetry Ingress

Changelog

Schema Release Rel37.20251016

  • advisory was implemented

Wiki Update 20241001

  • Update to be in line with schema addition
  • More information about telemetry
  • Most diagrams are now Mermaid code
  • Various smaller additions and corrections

Schema Release R36 20240930

  • "preheat" and "recovery" timestamps were added to the "scheduled" CloudEvent and the documentation was updated in that sense.
  • referencing a schema is now allowed in all payloads, making it easier to test payloads for compliance

Schema Release R35 20240807

  • Add "end" on last plan instruction

Wiki Update 20240807

  • Add "end" on last plan instruction
  • Fix "source" field description
  • Add zip naming.

Wiki Update 20240724

  • Fix the type in Started and Completed Event examples

Schema Release R34 20240721

  • remove SI unit for coulomb (prefer A for ampere)

Schema Release R33 20240719

  • add "generic" deviceType
  • add "generic" commandType
  • add "maximumCurrentLimit" commandType
  • add support for units in Amperes (A) & Coulombs (C) for Current Limiting
  • add Demand Response Schema overview page

Wiki Update 20240709

  • Add note about securing partner Web Hook endpoint

Wiki Update 20240703

  • Replace inline technical json schema docs with a downloadable zip file for clarity

Wiki Update 20240603

  • Publish command and control diagram in new page (draft)

Wiki Update 20240509

  • Add Telemetry Ingress information
  • Update logos, formatting
  • Update device plan hosting sample domain to be final name of connect.hiloenergie.com

Schema Release R32 20240318

  • Add support for deviceOptOut and plansUpdated subtypes for Amended
  • Additional title metadata
  • Some typos corrected

Contents

  1. Overview
  2. Subscribing to Demand Response Events
  3. Sequence Diagram for Publishing Events
  4. Demand Response High Level State Machine
  5. Hilo Demand Response Cloud Event Format
  6. Required CE Attributes
  7. Optional CE Attributes

Overview

Hilo Connect provides a standardized way for partners to integrate with Hilo’s Demand Response (DR) platform. It uses CloudEvents 1.0 for messaging and JSON schemas for payload validation. This documentation explains the event lifecycle, schema references, and integration steps.

Subscribing to Demand Response Events

Hilo offers delivery of DR events and associated data via a web hook callback to a web service provided by the integrating partner. The application layer messaging is encoded using CloudEvents 1.0 specification. The initial setup for the web hook delivery requires a one-time handshake according to the CloudEvents "abuse protection" specification; see: spec/http-webhook.md at v1.0 · cloudevents/spec (github.com)

This one-time subscription is typically performed during a conference call where an engineer on each side can coordinate and confirm a successful subscription.

Securing the partner Web Hook endpoint

We recommend at a minimum that the delivery endpoint require a HTTP header containing an "api key" in conjunction with TLS. Each message would be delivered accompanied by this HTTP header.

Sequence Diagram for Publishing Events

sequenceDiagram
    participant vpp as Hilo VPP
    participant eventgrid as Azure Event Grid
    participant storage as Azure Storage Account
    participant partner as Partner
    vpp->>+storage: Publish plans
    vpp->>+eventgrid: Publish CloudEvent
    eventgrid ->>+partner: HTTP Push CloudEvent
    Note right of storage: JSON DR event with SAS<br/> URL for storage account
    partner->>+storage: HTTP GET (SAS URL)
Loading

Demand Response High Level State Machine

  • This represents the state machine for a partner's DR event lifecycle - i.e. one lifecycle of [advisory, created, started, completed] per GD/DR, per partner, for the partner's entire fleet of DERs.
---
config:
  layout: elk
  elk:
    mergeEdges: false
    nodePlacementStrategy: NETWORK_SIMPLEX
---

flowchart LR
    start(((" "))) --> B("advisory")
    B --> C("scheduled") 
    C --> D("started")
    D --> E("completed")
    E --> F(((" ")))
    G --> F
    C --> H("amended")
    H --> G & D
    B --> G("canceled")
    H --> H
    
    
    style start stroke:#000000,fill:#000000
    style F stroke:#FFFFFF,color:#000000,fill:#000000,stroke-width:4px,stroke-dasharray: 0

    classDef notImplemented stroke:#757575,stroke-width:2px,stroke-dasharray: 2,color:#757575;
    class D,E,G notImplemented;

    classDef implemented stroke:#000000;
    class B,C,H implemented;
Loading

  • advisory messages are an early indication for a Demand Response event. If the event is confirmed, it will start at the specified start time and end at the specified end time. It does not guarantee that the event will take place and can be considered expired if no scheduled event is received by the start time.

  • scheduled messages are sent when the plans for a partner are ready for retrieval. This happens a few hours before the start of the event. The message contains 4 timestamps:

    • preheat: optional, indicates the first time for which an instruction might be sent for any device that participates in the event.

    • start: the start of the demand response event. This is were the participants should reduce their power consumption.

    • end: the end of the demand response event. This is were the participants can gradually pick their usual consumption back up.

    • recovery: optional, indicates the latest time for which an instruction might be sent for any device that participates in the event. At this point, normal consumption can resume on all devices.

  • amended can be of two types: deviceOptOut and plansUpdated. deviceOptOut might be sent any time, plansUpdated can be sent before an event. Each amendment generates a CE. deviceOptOut cancels plans for specific devices, while plansUpdated notifies the partner that they should refresh their copies of the plan(s)/schedule(s) from the secured endpoint.

Simplified diagram showing only currently supported events:

---
config:
  layout: elk
---
flowchart LR
    start(((" "))) --> B("advisory")
    B --> C("scheduled")
    C --> H("amended") & F(((" ")))
    H --> H & F
    style start stroke:#000000,fill:#000000
    style F stroke:#FFFFFF,color:#000000,fill:#000000,stroke-width:4px,stroke-dasharray: 0

    classDef implemented stroke:#000000;
    class C,H implemented;
Loading

Hilo Demand Response CloudEvents Format

Required CloudEvents Attributes

  • id

    • This is an autogenerated [Guid] for every instance of a CloudsEvents message

  • source

    • This will match the component responsible for dispatching the Cloud Event:

      • e.g. hilo-hiloconnect-<env>--<partner>-evgt

        • <env> could be dev, qa, preprod, prod etc.
        • <partner> identifies the specific integration

  • specversion: "1.0"

  • type

    • This attribute signifies the phase of the Demand Response lifecycle and should be used to prime the appropriate parser for the incoming Cloud Event.

      • com.hiloenergie.demand_response.event.advisory

        • uses data, datacontenttype, dataschema with embedded JSON

        • Advises subscribers that one or more Demand Response events might be scheduled by the specified start time. The data includes specifics on the planned events. Subscribers may use this time to inform customer support, prepare infrastructure for scaling, notify downstream customers etc.

      • com.hiloenergie.demand_response.event.scheduled

        • uses data, datacontenttype, dataschema with embedded JSON

        • uses subject with SAS URI to external plan JSON file

        • Advises subscribers that a Demand Response event has been scheduled, and the plan should be retrieved from the embedded URL.

      • com.hiloenergie.demand_response.event.amended

        • uses data, datacontenttype, dataschema with embedded JSON

        • SUBTYPE plansUpdated: Advises subscribers that a previously scheduled Demand Response event has been amended, and the updated plan should be retrieved from the embedded URL.

          • uses subject with SAS URI to external plan JSON file

        • SUBTYPE deviceOptOut: Advises subscribers that a user opted out a device from the planned event on the side of the DR program. Opt-outs from the subscriber side will not be transmitted (as they are already known by the party, which will inform the DR program through relevant telemetry messages). The payload is a JSON array of one or more string device identifiers. Individual opt-outs are bundled every minute in one message, if applicable.

      • com.hiloenergie.demand_response.event.canceled (Not supported yet)

        • uses data, datacontenttype, dataschema with embedded JSON

        • Advises subscribers that a previously scheduled Demand Response event has been canceled.

      • com.hiloenergie.demand_response.event.started (Not supported yet)

        • uses data, datacontenttype, dataschema with embedded JSON

        • Advises subscribers that a previously scheduled Demand Response event has started.

      • com.hiloenergie.demand_response.event.completed (Not supported yet)

        • uses data, datacontenttype, dataschema with embedded JSON

        • Advises subscribers that a previously scheduled Demand Response event has completed.

      • com.hiloenergie.demand_response.plan (Not supported yet)

        • uses data, datacontenttype, dataschema with embedded JSON

        • When the plan size is small (i.e. under 60k) and targets a single device, the plan may be embedded in the body of the CloudEvents message instead of using the subject field to point to an external zip file.

        • This may be enhanced to support multiple devices, as long as the total size of the CloudEvents message remains below 64kb

  • time

    • <rfc3339> format

    • This represents the time of creation of this CloudEvents message only. It is not related to the DR event scheduling.

  • datacontenttype

    • Specified as: "application/vnd.hiloenergie.demandresponse+json"

      • See: spec/cloudevents/formats/json-format.md at main · cloudevents/spec (github.com) (section 3.1.1)

  • dataschema

  • data

    • A JSON payload that conforms to the dataschema as specified above.

      • The JSON schema subtype will be one of #/$defs/EventScheduled, #/$defs/EventStarted, #/$defs/EventCompleted, #/$defs/EventCanceled, #/$defs/EventAmended or #/$defs/DevicePlan (Inline plan not supported yet)

Optional CloudEvents Attributes

If the message is of type [scheduled]{.ul} or [amended]{.ul} with subtype plansUpdated, there will be a URL to an external JSON plan file in the subject field. This pointer should be a time-limited (12 hours), IP whitelisted (if possible) SAS URI (see Grant limited access to data with shared access signatures (SAS) - Azure Storage | Microsoft Learn) for the plans associated with this event. One plan per device is generated, and all plans are zipped into one file. The external partner parses this event and retrieves the zip to store it securely on their side to parse and execute/schedule the individual files as required.

  • subject

    • Required for a type of com.hiloenergie.demand_response.event.[scheduled or]{.ul} com.hiloenergie.demand_response.event.[amended]{.ul}

      • With support for inline plans (when total message size is less than 64kb) this attribute is not required -- see com.hiloenergie.demand_response.plan above

    • For a Demand Response (DR) event that references a generated plan file (i.e. a zip stored in Azure Storage containing JSON plans), this value should be a dynamically generated SAS URI:

    • External Plan structure and JSON format:

      • Plan stored in a partner-specific folder in an azure storage account as a blob file (one file per device compressed as a zip)

      • The zip file is a flat structure containing one or more json files named in the format: "<partnername>-deviceplan-<normalized-deviceid>.json"

      • The zip file itself is named in the format (the timestamp is UTC): "plans-1969-10-05T10:55:00.zip

      • A manifest file will be included with the name "<partnername>-manifest.json" (Not supported yet)

        • The manifest will contain a list of all json device plan filenames, their sha1 hashes and (optionally) a digital signature created by a public key provided by the partner

Example Flow (Minimal)

This is an example flow of a scheduled Demand Response event from the Hilo Energie 2.0 DR API. This consists of a sequence of CloudEvents messages encoded using JSON:

  • Scheduled
{
    "specversion": "1.0",
    "type": "com.hiloenergie.demand_response.event.scheduled",
    "source": "hilo-hiloconnect-prod--partner-evgt",
    "id": "1234-5678-9012-3456",
    "time": "2023-10-11T13:00:00Z",
    "datacontenttype": "application/vnd.hiloenergie.demandresponse+json",
    "dataschema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "data": {
        "eventId": "38f3b3a8-b2a4-4bd0-9e84-a3c01b8d01b5",
        "lastModifiedDate": "2023-10-05T18:34:46Z",
        "body": {
            "event": "scheduled",
            "preheat": "2023-10-05T18:13:46Z",
            "start": "2023-10-05T18:34:46Z",
            "end": "2023-10-05T19:11:38Z",
            "recovery": "2023-10-05T19:36:42Z"
        }
    },
    "subject": "https://hilodemandresponseplan..."
}
  • Amended

An amendment type of deviceOptOut will include an array of device identifier strings in a devices field.

{
    "specversion": "1.0",
    "type": "com.hiloenergie.demand_response.event.amended",
    "source": "hilo-hiloconnect-prod--partner-evgt",
    "id": "1234-5678-9012-3456",
    "time": "2023-10-11T13:00:00Z",
    "datacontenttype": "application/vnd.hiloenergie.demandresponse+json",
    "dataschema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "data": {
        "eventId": "33f4e1ce-6198-4a4a-bd97-a2bd731c9f86",
        "lastModifiedDate": "2024-03-13T00:00:00Z",
        "body": {
            "event": "amended",
            "reason": {
                "amendment": "deviceOptOut",
                "devices": ["dev1", "dev2", "dev3"]
            }
        }
    }
}

An amendment type of plansUpdated acts like the initial scheduled message, where there will be a secured link to a zip file of new and/or amended plan files. Amended device plans completely replace any previous plans. It is not required to merge or patch the prior plan.

{
    "specversion": "1.0",
    "type": "com.hiloenergie.demand_response.event.amended",
    "source": "hilo-hiloconnect-prod--partner-evgt",
    "id": "1234-5678-9012-3456",
    "time": "2023-10-11T13:00:00Z",
    "datacontenttype": "application/vnd.hiloenergie.demandresponse+json",
    "dataschema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "data": {
        "eventId": "33f4e1ce-6198-4a4a-bd97-a2bd731c9f86",
        "lastModifiedDate": "2024-03-13T00:00:00Z",
        "body": {
            "event": "amended",
            "reason": {
                "amendment": "plansUpdated"
            }
        }
    },
    "subject": "https://hilodemandresponseplan..."
}
  • Started (Not supported yet)
{
    "specversion": "1.0",
    "type": "com.hiloenergie.demand_response.event.started",
    "source": "hilo-hiloconnect-prod--partner-evgt",
    "id": "1234-5678-9012-3456",
    "time": "2023-10-11T13:00:00Z",
    "datacontenttype": "application/vnd.hiloenergie.demandresponse+json",
    "dataschema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "data": {
        "eventId": "38f3b3a8-b2a4-4bd0-9e84-a3c01b8d01b5",
        "lastModifiedDate": "2023-10-05T18:34:46Z",
        "body": {
            "event": "started",
            "start": "2023-10-05T18:34:46Z"
        }
    }
}
  • Completed (Not supported yet)
{
    "specversion": "1.0",
    "type": "com.hiloenergie.demand_response.event.completed",
    "source": "hilo-hiloconnect-prod--partner-evgt",
    "id": "1234-5678-9012-3456",
    "time": "2023-10-11T13:00:00Z",
    "datacontenttype": "application/vnd.hiloenergie.demandresponse+json",
    "dataschema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "data": {
        "eventId": "38f3b3a8-b2a4-4bd0-9e84-a3c01b8d01b5",
        "lastModifiedDate": "2023-10-05T18:34:46Z",
        "body": {
            "event": "completed",
            "end": "2023-10-05T20:34:46Z"
        }
    }
}
  • Canceled (Not supported yet)

These messages are delivered in this order to the configured Web Hook of an external partner. The delivery is managed by Azure Event Grid, and each Web Hook invocation is authenticated over HTTPS and an "API key" sent via a HTTP header. This API key is provided by the external partner and is transferred to Hilo Energie securely via an independent process. The initial subscription of the Web Hook endpoint requires a one-time handshake process to prevent rogue subscriptions to the endpoint.

Individual Device Plan (example)

A tool to generate examples is available in the Github repository (plantool.ps1)

{
    "$schema": "https://schema.hiloenergie.com/json/v1/demandresponse.schema.json",
    "eventId": "38f3b3a8-b2a4-4bd0-9e84-a3c01b8d01b5",
    "lastModifiedDate": "2023-10-11T18:29:09Z",
    "body": {
        "deviceId": "tstat-0x00000001",
        "commandType": "heatingSetpointDelta",
        "deviceType": "thermostat",
        "commands": [
            {
                "parameter": {
                    "unit": "C",
                    "value": 2
                },
                "start": "2023-10-12T04:05:00Z"
            },
             {
                "parameter": {
                    "unit": "C",
                    "value": 3
                },
                "start": "2023-10-12T04:10:00Z"
            },
             {
                "parameter": {
                    "unit": "C",
                    "value": -2
                },
                "start": "2023-10-12T04:15:00Z"
            },
             {
                "parameter": {
                    "unit": "C",
                    "value": -3
                },
                "start": "2023-10-12T04:20:00Z",
                "end": "2023-10-12T04:25:00Z"
            }
        ]
    }
}

One plan uses the same commandType throughout, while units may vary within that same commandType; for example, maximumPowerLimit could be controlled in watt or in percent, depending on the controlled device. The deviceType propery is purely informational and does not need to be used. The command array is not limited in terms of the amount of commands, although not more than 12 should be used.

Telemetry Ingress

Overview

Telemetry ingress is accepted via HTTP POST to an OAuth 2.0 secured endpoint. The messages are expected to in the JSON format, though support may arrive in the future for Avro and ProtoBuf structures too. The JSON may be GZip compressed, if desired. The JSON may be in any format native to the external partner, though a JSON schema definition would be welcome to help validate and debug telemetry parsing issues on our side.

Authentication

Authentication is performed using the OAuth 2.0 Client Credentials flow, whereby a ClientId and a ClientSecret are provided to the partner. The secret is in the form of a secret string that must be kept secure. In the future Hilo may move to an X509 certificate-based secret, but this is not envisaged for 2024/2025. Details of the token endpoint and parameters will be shared with partners via a secure means.

Endpoint

The endpoint FQDN will take the form of https://partnername[.dev|.qa].hiloenergie.com/ingress (?). Each HTTP POST message should contain the JSON message as-is in the body with a standard Content-Type: application/json; charset=utf-8 header, assuming UTF-8 is employed. The bearer token should be sent via the standard header Authorization with a value of Bearer ACCESSTOKEN. If compressed JSON is being delivered, supply a header of Content-Encoding: gzip.

Batch support

For a single message, send a single root JSON object:

{
  ...
}

For sending a batch of JSON messages, encode as an array in the body:

[
  { ... },
  { ... },
  { ... }
]

Message types

Expected message types would be device lifecycle events (pairing/unpairing, etc.) and explicit DR modes (started, ended, opt-out). Setpoint and status changes are welcomed. The partner is expected to provide details on the message format and Hilo will write a translation layer to parse incoming messages.

Expected messages:

  • Device life cycle events

    • pair / unpair

  • Demand Response Event participation

    • Started
    • Ended
    • Opt-out (from the device / partner app)

Optional:

  • Instant power (or current status, on / off, depending on device capabilities)
  • Thermostats:
    • Ambient temperature & setpoint
    • Temperature format
    • if applicable Thermostat mode
      • ambient / floor temperature etc.
      • state (cool / heat etc.)
      • minimum / maximum configured temperature

Expected HTTP response

All messages received via the endpoint return a HTTP 202 Accepted. This is a response status code that indicates that the request has been accepted for processing, but the processing has not been completed. Diagnostic information about unrecognized or malformed messages is not immediately available to partners but is logged within Hilo and may be shared with partner developers over conventional channels (such as a Slack/Teams channel, or email.)

Clone this wiki locally