Synthetic notifications #276
Conversation
|
Hi @m-glz,
|
|
Hi, @pbruenn
I will also try to prepare some code examples of use cases for the new notification types. |
|
Thanks for clarification. Yes, the examples are important for me to understand this and get a better idea of how we can cleanly implement this. Don't put to much effort into cleanup just yet. I first want to try your example to see in which direction we should move this. |
|
Here is a small sample for the use of the I will add a sample for // This is a sample application demonstarting the use of the proposed NOTIFY_CONNECTION_LOST synthetic notification
//
// This sample requires C++23
//
// This application will connect to a TwinCAT PLC and output the PLC state. If the connection drops, the application
// will attempt to reconnect once a until successful.
//
// To simplify the code, error handling has been largely omitted.
//
// Without the NOTIFY_CONNECTION_LOST the application would never detect the connection loss, because it does not
// send any commands, and thus never gets any errors. With the NOTIFY_CONNECTION_LOST notification, the connection
// loss is detected, and the connection reestablished.
//
// To test:
//
// - Start the application and connect to a TwinCAT PLC instance. The application will start logging the device state.
// - Start and stop the PLC to verify that the device state is logged
// - Put TwinCAT into config mode. This will cause TwinCAT to drop the connection.
// - Put TwinCAT back into run mode. The application will reestablish the connection and resume logging the device state.
// - Start and stop the PLC to verify that the device state is logged again
//
// Without the NOTIFY_CONNECTION_LOST notification, the connection will never be reestablished, and logging of the PLC state
// will never resume even after TwinCAT has been put back into run mode.
#include <AdsLib.h>
#include <chrono>
#include <cstddef>
#include <cstdlib>
#include <iostream>
#include <semaphore>
#include <span>
#include <thread>
using namespace std::literals;
// A semaphore used to communicate with the main thread
std::binary_semaphore connectionLostSemaphore { 0 };
// Callback for connection loss notifications
void connectionLostCallback(const AmsAddr *amsAddress, std::uint32_t type, std::uint32_t userValue)
{
// Log the connection loss
std::cout << "Connection loss detected." << std::endl;
// Notify the main thread
connectionLostSemaphore.release();
}
// Callback for device state notifications
void deviceStateCallback(const AmsAddr *amsAddress, const AdsNotificationHeader *header, std::uint32_t userValue)
{
// Sanity check
if (header->cbSampleSize != sizeof(std::uint16_t))
{
std::cout << "Received invalid device state with size " << header->cbSampleSize << std::endl;
return;
}
// Get the value
std::uint16_t state;
std::memcpy(&state, header + 1, sizeof(state));
if constexpr (std::endian::native != std::endian::little)
{
state = std::byteswap(state);
}
// Log the state
std::cout << "Device state: " << state << std::endl;
}
auto main(int argc, char *argv[]) -> int
{
// Decode the parameters
if (argc < 4)
{
std::cerr << "Usage: <AMS Net-ID> <AMS Port> <IP address>" << std::endl;
return EXIT_FAILURE;
}
AmsAddr amsAddress { .netId { argv[1] }, .port { std::uint16_t(std::atoi(argv[2])) } };
const char *ipAddress { argv[3] };
// Keep reconnecting forever
for (;;)
{
//////////////////////////////////////////////////
// Establish the connection
// Try to connect
std::cout << "Initiating connection..." << std::endl;
auto connectionError = AdsAddRoute(amsAddress.netId, ipAddress);
// Check for error
if (connectionError != ADSERR_NOERR)
{
std::cout << "Connection failed: 0x" << std::hex << connectionError << std::dec << std::endl;
// Sleep for 1s and retry
std::this_thread::sleep_for(1s);
continue;
}
// Open a port
auto port = AdsPortOpenEx();
//////////////////////////////////////////////////
// Add handlers and service the connection
// Add a handler for the connection loss notification
std::uint32_t connectionLossNotification;
AdsAddSyntheticDeviceNotificationReqEx(
port, &amsAddress, NOTIFY_CONNECTION_LOST, connectionLostCallback, 0, &connectionLossNotification);
// Add a handler for the PLC state
const AdsNotificationAttrib deviceStateNotificationAttrib { .cbLength = sizeof(std::uint16_t), .nTransMode = ADSTRANS_SERVERONCHA };
std::uint32_t deviceStateNotification;
auto addNotificationError = AdsSyncAddDeviceNotificationReqEx(port,
&amsAddress,
ADSIGRP_DEVICE_DATA,
ADSIOFFS_DEVDATA_ADSSTATE,
&deviceStateNotificationAttrib,
deviceStateCallback,
0,
&deviceStateNotification);
// The connection might have failed before the call to AdsAddSyntheticDeviceNotificationReqEx, so check for an error here as well
if (addNotificationError == ADSERR_NOERR)
{
std::cout << "Connected." << std::endl;
// Wait for the connection to terminate
connectionLostSemaphore.acquire();
}
else
{
std::cout << "AdsSyncAddDeviceNotificationReqEx failed: 0x" << std::hex << addNotificationError << std::dec << std::endl;
}
//////////////////////////////////////////////////
// Shut down the connection and clean up
std::cout << "Shutting down connection." << std::endl;
// Clean everything up before trying to re-add the route
AdsSyncDelDeviceNotificationReqEx(port, &amsAddress, deviceStateNotification);
AdsDelSyntheticDeviceNotificationReqEx(port, &amsAddress, connectionLossNotification);
AdsPortCloseEx(port);
AdsDelRoute(amsAddress.netId);
// Clean up any dangling connection loss notifications
void(connectionLostSemaphore.try_acquire());
// Sleep for 1s before reconnecting
std::this_thread::sleep_for(1s);
}
} |
|
Ok, here is a sample for the use NOTIFY_NOTIFICATION_RCV. This application just logs all the notifications from a single AoE frame to standard out as a block. // This is a sample application demonstarting the use of the proposed NOTIFY_NOTIFICATION_RCV synthetic notification
//
// This sample requires C++23
//
// This application will connect to a TwinCAT PLC and register notifications for symbols specified on the command
// line. It will then group change notifications into transactions, where each transaction contains all the changes
// from a single DEVICE_NOTIFICATION AoE frame.
//
// To simplify the code, error handling has been largely omitted.
//
// Without the NOTIFY_NOTIFICATION_RCV, it would not be possible to group the change notifications together. This
// could introduce a critical bottleneck in situations where a lot of PLC variables change at the same time.
//
// To test:
//
// - Start the application and connect to a TwinCAT PLC instance.
// - Perform an action in the PLC that will change multiple of the watched symbols within the same cycle
// - Verify that the changes are grouped into a single transaction.
#include <AdsLib.h>
#include <compare>
#include <cstdlib>
#include <iomanip>
#include <iostream>
#include <span>
#include <string>
#include <utility>
#include <vector>
// The symbol names, in the order they were specified on the command line
std::vector<std::string> symbolNames;
// Information about a symbol change
struct ChangeInfo
{
std::size_t symbolIndex;
std::vector<std::byte> data;
};
// The list of changes since the last transaction
std::vector<ChangeInfo> changeLog;
// Callback for change notifications
void symbolChangeCallback(const AmsAddr *amsAddress, const AdsNotificationHeader *header, std::uint32_t index)
{
// Sanity check
if (std::cmp_greater_equal(index, symbolNames.size()))
{
std::cout << "Received invalid notification with index " << index << std::endl;
return;
}
// Extract the data
std::span data { reinterpret_cast<const std::byte *>(header + 1), std::size_t(header->cbSampleSize) };
// Add an entry to the log
changeLog.push_back({ .symbolIndex { std::size_t(index) }, .data { std::from_range, data } });
}
// Callback for notification received notifications
void notificationReceivedCallback(const AmsAddr *amsAddress, std::uint32_t type, std::uint32_t userValue)
{
// This callback just outputs the list of changes to std::cout. A real-life application could do
// any of the following action, e.g.:
//
// - Write the changes to a database using a single database transaction
// - Send the changes to a REST interface using a single HTTP POST request
// - Send the changes to a UI thread using a single event
// - etc. etc. etc.
//
// Each of these actions benefit heavily from batching together multiple changes. When dealing with a
// large number of symbols, having to process each change individually could introduce a critical
// performance bottleneck.
std::cout << "BEGIN NOTIFICATION FRAME\n[";
for (bool isFirst = true; auto &&[symbolIndex, data] : changeLog)
{
if (!std::exchange(isFirst, false))
{
std::cout << ",";
}
std::cout << "\n {\n \"symbol\": \"" << symbolNames[symbolIndex] << "\",\n \"data\": ";
char prefix { '"' };
for (auto &&byte : data)
{
std::cout << std::exchange(prefix, ':') << std::hex << std::setfill('0') << std::setw(2) << static_cast<unsigned int>(byte) << std::dec << std::setfill(' ');
}
std::cout << "\"\n }";
}
std::cout <<"\n]\nEND NOTIFICATION FRAME" << std::endl;
changeLog.clear();
}
auto main(int argc, char *argv[]) -> int
{
// Decode the parameters
if (argc < 5)
{
std::cerr << "Usage: <AMS Net-ID> <AMS Port> <IP address> <symbol name>..." << std::endl;
return EXIT_FAILURE;
}
AmsAddr amsAddress { .netId { argv[1] }, .port { std::uint16_t(std::atoi(argv[2])) } };
const char *ipAddress { argv[3] };
for (int index = 4; index < argc; ++index)
{
symbolNames.emplace_back(argv[index]);
}
std::cout << "Press Enter to exit.\nLog:" << std::endl;
// Establish the connection
AdsAddRoute(amsAddress.netId, ipAddress);
// Open a port and add the notification received callback
auto port = AdsPortOpenEx();
// Add a handler for the notification received notification
std::uint32_t notificationReceivedNotification;
AdsAddSyntheticDeviceNotificationReqEx(
port, &amsAddress, NOTIFY_NOTIFICATION_RCV, notificationReceivedCallback, 0, ¬ificationReceivedNotification);
// Remember all the notifications and handles, so we can release them
struct SymbolInfo
{
std::uint32_t handle;
std::uint32_t notification;
};
std::vector<SymbolInfo> symbolInfos;
// Add notifications for all the symbols
for (std::size_t index = 0; index < symbolNames.size(); ++index)
{
SymbolInfo symbolInfo;
// Get a handle
auto &&symbolName { symbolNames[index] };
std::uint32_t bytesRead;
auto lookupError = AdsSyncReadWriteReqEx2(port,
&amsAddress,
ADSIGRP_SYM_HNDBYNAME,
0,
sizeof(symbolInfo.handle),
&symbolInfo.handle,
std::uint32_t(symbolName.size()),
symbolName.data(),
&bytesRead);
if (lookupError != ADSERR_NOERR)
{
std::cerr << "Error looking up symbol \"" << symbolName << "\": 0x" << std::hex << lookupError << std::dec << std::endl;
continue;
}
// Add a notification for it
const AdsNotificationAttrib attrib { .cbLength = sizeof(std::uint16_t), .nTransMode = ADSTRANS_SERVERONCHA };
AdsSyncAddDeviceNotificationReqEx(port,
&amsAddress,
ADSIGRP_SYM_VALBYHND,
symbolInfo.handle,
&attrib,
symbolChangeCallback,
std::uint32_t(index),
&symbolInfo.notification);
symbolInfos.push_back(symbolInfo);
}
// Wait for a key press
std::cin.ignore();
// Clean everything up before exiting
for (auto &&[handle, notification] : symbolInfos)
{
AdsSyncDelDeviceNotificationReqEx(port, &amsAddress, notification);
AdsSyncWriteReqEx(port,
&amsAddress,
ADSIGRP_SYM_RELEASEHND,
0,
sizeof(handle),
&handle);
}
AdsDelSyntheticDeviceNotificationReqEx(port, &amsAddress, notificationReceivedNotification);
AdsPortCloseEx(port);
AdsDelRoute(amsAddress.netId);
return EXIT_SUCCESS;
} |
|
Thanks a lot. Sorry for not responding more early. It was a short and busy week :-(. I struggle with the second usecase RCV. If I understand that correctly: The PLC sends a large notification with many different symbols changed. When we receive the notification and processing it we create a synthetic notification for each symbol and then process each of the symbol changes individually. How is this faster then just splitting the processing in the normal notification handler? We only delay it by the time we need too complete the entire notification. Again, iam not an ADS expert but I have a feeling your problem could be mitigated by tuning I still like the LOST notification very much. I will try to integrate this within the next few days. |
|
Hi Patrick, There is only one single synthetic notification per frame. TwinCAT will take all the variables that changed in one single PLC cycle and put them into a single Network packet with a single AoE header. So, If variables MAIN.var1, MAIN.var2 and MAIN.var3 change in the same cycle, only a single network packet is sent, that looks something like this (simplified):
This is nothing new, that is how TwinCAT works, and that is how the ADS protocol is designed. This is done, of course, because placing all the change notification together into a single packet greatly reduces network traffic. The existing library, however, will generate only the following notifications for that:
Now, if we want to do some batch processing on the notifications, there currently is no way for us to know that MAIN.var3 is the last notification, and that we can now go ahead and process the changes. For all we know, there could be a fourth change notification in the AoE packet. So all we can do is wait around for a bit to see if maybe the callback gets called a fourth time, and do our processing only after nothing has happened for a certain amount of time. Our changes merely add a final synthetic notification that tells the caller that notifications for all the samples in the packet have been sent, and that it can go ahead and process them now. So the notifications for the same AoE packet in the updated library would look like this:
Now, when we get It is important to understand that this changes nothing in the ADS protocol, or in the way TwinCAT or the notifications work. It is merely an extra callback in the client library to tell the caller that all the samples in the received frame have been processed.
From an implemetation standpoint, Batch processing of changes is very important for applications that need to log or forward changes. Without the possibility of handling changes as a batch, many applications are simply not possible. One of our use-cases, for example, is sending data to an InfluxDB time series database using their HTTP write_lp API. Using |
|
Okay, I think I now understand your usecase. But I have more questions. Wouldn't it be more efficient to fill the InfluxDB directly from the PLC? Like its done here: Or even use the TwinCAT Analytics Storage Provider: Did you consider these TwinCAT functions? |
|
That was just an example usecase, demonstrating why this functionality is important, and how it would be applied. I'm not suggesting that we are stuck on this particular issue, and that we are in search of a specific solution for that specific problem. What we want to be able to do more generally, is get data from the PLC in an efficient way, and then process it in a number of different ways, only one of which is sending it to InfluxDB. For us, and for anyone else who has to deal with large amounts of data, it is crucial to be able to eliminate data bottlenecks, and batch processing is the first and most important step in accomplishing that. That is why ADS itself sends change notifications in batches, and that is why the ADS protocol offers things like |
3 participants
Overview
This pull request introduces synthetic notifications as a new notification channel in the ADS library.
Unlike standard ADS notifications that originate from target ADS devices, synthetic notifications are generated within the library to inform users about key internal events and states.
Implemented Synthetic Notifications
Two new synthetic notification types have been added:
NOTIFY_CONNECTION_LOST
This synthetic notification type is emitted when the connection to a target ADS device is lost.
Previously, client code could only detect a connection failure indirectly by sending a request and receiving an error response. For applications that rely mainly on ADS notifications to track symbol updates (without performing regular read/write operations), this made it impossible to detect connection loss at runtime.
With NOTIFY_CONNECTION_LOST, the library now notifies registered users whenever any socket operation for a given connection fails, allowing client code to react immediately to connection issues.
NOTIFY_NOTIFICATION_RCV
This synthetic notification type is emitted whenever any ADS notification is received from a target device.
It serves as a lightweight, data-free event that is triggered after the invocation of all ADS notifications contained within an AMS packet, enabling client code to receive a general “summary” notification when any subscribed ADS symbol is updated.
This can be useful in scenarios where multiple symbols are updated via ADS notifications, and a single trigger is needed to perform follow-up actions, such as refreshing a GUI, aggregating data, or committing a database transaction.