Issue when updating from v3 to v4

[rwuthric]

We updated our Agents from version 2.3.0.4 to 2.4.0.7 (we use the Docker image built from this repository).

In our case the updated Agent seems no longer to understand or device.xml files and does no longer map the DataItems.

Here the device.xml we use (a minimal version to illustrate the issue we face):

<MTConnectDevices>
  <Devices>
    <Device uuid="MINIMAL" id="minimal" name="Minimal-Device">
      <Description>
         A minmal device to test the agent
      </Description>
      <DataItems>
        <DataItem category="EVENT" id="avail" name="avail" type="AVAILABILITY"/>
      </DataItems>
    </Device>
  </Devices>
</MTConnectDevices>

The agent connects to the adapter, but when the adapter sends the SHDR message |avail|AVAILABLE, we get this error in the Agent version 2.4.0.7:

[info] Connector::connected: Connected with: 10.0.0.229:7878
[info] Connector::reader->Connector::parseSocketBuffer->Connector::processLine->ShdrAdapter::processData->DataItemMapper.ShdrTokenMapper.operator->DataItemMapper.ShdrTokenMapper.mapTokensToDataItem: Could not find data item: avail

Using the Agent version 2.3.0.4 everything works fine as expected.

Do we have an error in our device.xml (and accidentally it just worked fine so far) ? Or did the SHDR protocol change with version 2.4.0.7 ? Or do we have to change something in our agent.cfg file?

For reference this is the agent.cfg we use:

Devices = device.xml
SchemaVersion = 2.4

Port = 5000

Adapters {

    Adapter {
       Host   = 10.0.0.229
       Port   = 7878
    }

}

Files {
    schemas {
        Path = /mtconnect/data/schemas
        Location = /schemas/
    }
    styles {
        Path = /mtconnect/data/styles
        Location = /styles/
    }
    Favicon {
        Path = /mtconnect/data/styles/favicon.ico
        Location = /favicon.ico
    }
}

DevicesStyle {
    Location = /styles/styles.xsl
}
StreamsStyle {
    Location = /styles/styles.xsl
}

logger_config
{
    logging_level = info
    output = file /home/agent/mtc_agent.log
}

[wsobel]

Looks like it should work. I'll try to reproduce the issue and fix the problem. I don't remember any changes to the tokenizing in 2.4, but something may be having a side effect.

[rwuthric]

Thank you very much
Here the image (public) of the agent that we created from this repo:
ghcr.io/demo-smart-factory-concordia-university/mtcagent
(workflow is here: https://github.com/Demo-Smart-Factory-Concordia-University/OpenFactory/blob/main/.github/workflows/mtc_agent.yml)

[wsobel]

Can you please include the results of the REST probe of the MTConnect agent? I want to make sure we are seeing the same response document.

Thanks
W

[rwuthric]

Here is the /probe endpoint:

<MTConnectDevices xmlns:m="urn:mtconnect.org:MTConnectDevices:2.4" xmlns="urn:mtconnect.org:MTConnectDevices:2.4" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mtconnect.org:MTConnectDevices:2.4 http://schemas.mtconnect.org/schemas/MTConnectDevices_2.4.xsd">
  <Header creationTime="2024-12-23T14:14:20Z" sender="8238b91bc439" instanceId="1734963210" version="2.4.0.7" deviceModelChangeTime="2024-12-23T14:13:30.940313Z" assetBufferSize="1024" assetCount="0" bufferSize="131072" />
  <Devices>
    <Agent hash="NPopf0HYmeP6eeBvINFPq2qiwyE=" id="agent_MINIMAL" mtconnectVersion="2.4" name="Agent" uuid="MINIMAL-AGENT">
      <DataItems>
        <DataItem category="EVENT" id="agent_avail" type="AVAILABILITY" />
        <DataItem category="EVENT" id="device_added" type="DEVICE_ADDED" />
        <DataItem category="EVENT" id="device_removed" type="DEVICE_REMOVED" />
        <DataItem category="EVENT" id="device_changed" type="DEVICE_CHANGED" />
        <DataItem category="EVENT" discrete="true" id="agent_MINIMAL_asset_chg" type="ASSET_CHANGED" />
        <DataItem category="EVENT" id="agent_MINIMAL_asset_rem" type="ASSET_REMOVED" />
        <DataItem category="EVENT" id="agent_MINIMAL_asset_count" representation="DATA_SET" type="ASSET_COUNT" />
      </DataItems>
      <Components>
        <Adapters id="__adapters__">
          <Components>
            <Adapter id="_127.0.0.1_7878" name="127.0.0.1:7878">
              <DataItems>
                <DataItem category="EVENT" id="_127.0.0.1_7878_connection_status" type="CONNECTION_STATUS" />
                <DataItem category="EVENT" id="_127.0.0.1_7878_adapter_uri" type="ADAPTER_URI">
                  <Constraints>
                    <Value>shdr://127.0.0.1:7878</Value>
                  </Constraints>
                </DataItem>
                <DataItem category="SAMPLE" id="_127.0.0.1_7878_observation_update_rate" statistic="AVERAGE" type="OBSERVATION_UPDATE_RATE" units="COUNT/SECOND" />
                <DataItem category="SAMPLE" id="_127.0.0.1_7878_asset_update_rate" statistic="AVERAGE" type="ASSET_UPDATE_RATE" units="COUNT/SECOND" />
                <DataItem category="EVENT" id="_127.0.0.1_7878_adapter_software_version" type="ADAPTER_SOFTWARE_VERSION" />
                <DataItem category="EVENT" id="_127.0.0.1_7878_mtconnect_version" type="MTCONNECT_VERSION" />
              </DataItems>
            </Adapter>
          </Components>
        </Adapters>
      </Components>
    </Agent>
    <Device hash="D0sfcooDdd+bcOLpZo9BNnDmJ1o=" id="minimal" name="Minimal-Device" uuid="MIN-001">
      <Description>A minmal device to test the agent</Description>
      <DataItems>
        <DataItem category="EVENT" id="avail" name="avail" type="AVAILABILITY" />
        <DataItem category="EVENT" discrete="true" id="minimal_asset_chg" type="ASSET_CHANGED" />
        <DataItem category="EVENT" id="minimal_asset_rem" type="ASSET_REMOVED" />
        <DataItem category="EVENT" id="minimal_asset_count" representation="DATA_SET" type="ASSET_COUNT" />
      </DataItems>
    </Device>
  </Devices>
</MTConnectDevices>

[wsobel]

Thanks. I wanted to make sure the probe reflected the input device model correctly. Looks like it is a bug.

[wsobel]

Tried to reproduce in development environment with no success. I will next use the docker environment to reproduce exactly as you have described. If possible I'll build with debug and test.

[rwuthric]

If you want to have exactly same environment as we have here the docker-compose.yml file:

services:

  external_agent:
    image: ghcr.io/demo-smart-factory-concordia-university/mtcagent
    container_name: minimal_agent
    environment:
      MTC_AGENT_UUID: MINIMAL-AGENT
      ADAPTER_IP: <put here you adapter IP address>
      ADAPTER_PORT: 7878
      ADAPTER_UUID: MIN-001
    ports:
      - 5000:5000
    volumes:
      - ./config:/mtconnect/config

and the docker compose project is organized as:

.
├── config
│   ├── agent.cfg
│   └── device.xml
└── docker-compose.yml

with config/agent.cfg

AgentDeviceUUID = $MTC_AGENT_UUID
Devices = device.xml
SchemaVersion = 2.4

Port = 5000

Adapters {

    Adapter {
       UUID   = $ADAPTER_UUID
       Host   = $ADAPTER_IP
       Port   = $ADAPTER_PORT
    }

}

Files {
    schemas {
        Path = /mtconnect/data/schemas
        Location = /schemas/
    }
    styles {
        Path = /mtconnect/data/styles
        Location = /styles/
    }
    Favicon {
        Path = /mtconnect/data/styles/favicon.ico
        Location = /favicon.ico
    }
}

DevicesStyle {
    Location = /styles/styles.xsl
}
StreamsStyle {
    Location = /styles/styles.xsl
}

logger_config
{
    logging_level = info
    output = file /home/agent/mtc_agent.log
}

and config/device.xml:

<MTConnectDevices>
  <Devices>
    <Device uuid="MINIMAL" id="minimal" name="Minimal-Device">

      <!-- General information -->
      <Description>
         A minmal device to test the agent
      </Description>
      <DataItems>
        <DataItem category="EVENT" id="avail" name="avail" type="AVAILABILITY"/>
      </DataItems>
    </Device>
  </Devices>
</MTConnectDevices>

The command

docker compose up -d

will build and deploy the agent by pulling our image we use. Or you can build the image as in our workflow and then replace the entry image in the docker-compose.yml with your image you built.

[wsobel]

Thanks, I was building the image on my local machine to begin a simple test. This is very helpful. I need to go in a minute to an appointment, but I'll try to steal some time during the holiday to work on this. I'll use your compose file to be as close to your issue as possible.

[wsobel]

I will need to rebuild since I'm on an ARM platform. Are you building the Ubuntu image from the docker file under the docker directory?

[rwuthric]

Yes exactly. It uses this Dokerfile from the repo: docker/ubuntu/Dockerfile.

We build it on the latest version of Ubuntu.

[wsobel]

I was able to reproduce the issue. The problem is with the device name and the UUID change in the configuration file. If I make this change it works:

Adapters {

    Adapter {
#       UUID   = $ADAPTER_UUID
       Host   = $ADAPTER_IP
       Port   = $ADAPTER_PORT
    }

}

The problem is that the Default device UUID comes from the device before the adapter is loaded, and the UUID changes. This issue relates to the order of operations when loading the configuration file. The specification of the UUID in the agent.cfg is somewhat of an anomaly and probably worked before because we were using names instead of UUIDs to reference the device. The change was made to support dynamic device updates and fix another bug related to on-the-fly name changes.

I'll see how to fix the issue–it may require some surgery in the source instantiation process and the reassignment of UUIDs (which are not supposed to change) or make it a two-pass since the device is reindexed when the UUID changes.

Anyway, the simple workaround is to comment out the UUID line for now, and it will work.

Do you need to change the UUID of the device or can you do that in the device.xml file?

[wsobel]

I may have an easy fix... will test it out.

[wsobel]

Fixed. Will create pull request.