[Feature Request] Implement last-delivered tracking and replay for mqtt-relay module

[virajdere]

@PatrickRitchie
Description:

When the MTConnect agent's durable: true setting is enabled, the mqtt-relay module should be able to replay all backlogged observations and assets from the agent's persistent buffer upon network reconnection or agent restart. The replay should start from the last observation that was successfully delivered to the MQTT broker, ensuring no data is lost or duplicated.

Problem Description:

Our current configuration uses the durable: true setting to ensure data is persisted to disk during network outages. The agent successfully receives data from a local SHDR adapter and stores it in the buffer folder. However, when the network connection is restored (or after the agent is restarted), the mqtt-relay module does not read this backlogged, persisted data. It only begins relaying new observations as they arrive. This behavior leads to a critical loss of historical data, despite the agent's buffer successfully retaining it on the file system.

This disconnect between the agent's persistence mechanism and the mqtt-relay's relaying behavior creates a significant gap in our data collection strategy.

Steps to Reproduce:

Start the TrakHound MTConnect .NET agent with the provided configuration.

Disconnect the agent's network connection.

Allow the connected SHDR adapter to send data to the agent for a period of time.

Reconnect the network connection.

Observe the data stream on the MQTT broker (Greengrass gateway).

Expected Behavior:

Upon reconnection, the mqtt-relay module should:

Read the last successfully delivered observation's sequence number from its own persistent state.

Read all observations from the agent's durable buffer, starting from that sequence number.

Publish this historical data to the MQTT broker in the correct sequence.

Once the backlog is cleared, transition to relaying new, real-time observations.

This would ensure a complete and unbroken data stream, fulfilling the promise of the durable: true setting.

Current Behavior:

The mqtt-relay module only relays new observations that are generated after the network connection is restored. The backlogged data, which is successfully stored on disk by the durable setting, is never sent.

[virajdere]

Feature available now: https://github.com/TrakHound/MTConnect.NET/releases/tag/v6.8.0