PLC4X OPC UA connection succeeds, but no data sent to Kafka #331
Description
Hello everyone,
I tried to connect to a physical OPC UA server using PLC4X version 0.12.0 and configured a Kafka source connector using the JSON configuration file attached below.
The connector starts without any error, and the logs indicate that the connection to the OPC UA server is successfully established. However, I don't see any data being published to Kafka, and the topic I specified is not being created.
I also connected to the same OPC UA server using UA Expert, and I can see the data updating in real-time, so I’m sure the server is working correctly.
Here's the configuration I used and the corresponding logs.
Could anyone help me understand what might be going wrong?
Thank you in advance!
code JSON :
{
"connector.class": "org.apache.plc4x.kafka.Plc4xSourceConnector",
"default.topic": "opcTest",
"jobs": "job",
"jobs.job.interval": "1000",
"jobs.job.tags": "Poid",
"jobs.job.tags.Poid": "ns=2;s=Studio.Tags.Application.poid",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter.schemas.enable": "false",
"sources": "opcserver",
"sources.opcserver.bufferSize": "1000",
"sources.opcserver.connectionString": "opcua:tcp://IP_adress:Port?discovery=false",
"sources.opcserver.jobReferences": "job",
"sources.opcserver.pollReturnInterval": "5000",
"tasks.max": "1",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter.schemas.enable": "false"
}
I have this log :
[2025-04-19 13:29:57,990] INFO initializing Kafka metrics collector (org.apache.kafka.common.telemetry.internals.KafkaMetricsCollector)
[2025-04-19 13:29:57,997] INFO These configurations '[metrics.context.connect.kafka.cluster.id, metrics.context.connect.group.id]' were supplied but are not used yet. (org.apache.kafka.clients.producer.ProducerConfig)
[2025-04-19 13:29:57,997] INFO Kafka version: 7.8.2-ccs (org.apache.kafka.common.utils.AppInfoParser)
[2025-04-19 13:29:57,997] INFO Kafka commitId: 753ac432ef38a79b7f27781cd77b656d5ffc2e8e (org.apache.kafka.common.utils.AppInfoParser)
[2025-04-19 13:29:57,997] INFO Kafka startTimeMs: 1745069397997 (org.apache.kafka.common.utils.AppInfoParser)
[2025-04-19 13:29:57,998] INFO [Worker clientId=connect-kafka-connect:8083, groupId=compose-connect-group] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
[2025-04-19 13:29:57,998] INFO AbstractConfig values:
bufferSize = 1000
connection-name = opcserver
connectionString = opcua:tcp://IP_adresse:Port?discovery=false
pollReturnInterval = 5000
queries = [job|opcTest|1000|Poid#ns=2;s=Studio.Tags.Application.poid_PES03]
(org.apache.kafka.common.config.AbstractConfig)
[2025-04-19 13:29:57,999] INFO Instantiating new PLC Driver Manager with class loader org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader@7ec7ffd3 (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:57,999] INFO Registering available drivers... (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,000] INFO [Producer clientId=connector-producer-opc-0] Cluster ID: rQ0nYkKkTByRU84BZboN9Q (org.apache.kafka.clients.Metadata)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol ab-eth (Allen Bradley ETH) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol ads (Beckhoff TwinCat ADS) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol bacnet-ip (BACnet/IP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol c-bus (Clipsal C-Bus) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol genericcan (Generic CAN) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol canopen (CAN open) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol eip (EthernetIP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol logix (Logix CIP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol firmata (Firmata) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol iec-60870-5-104 (IEC 60870-5-104) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol knxnet-ip (KNXNet/IP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol open-protocol (Open-Protocol) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol modbus-ascii (Modbus ASCII) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol modbus-rtu (Modbus RTU) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol modbus-tcp (Modbus TCP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol opcua (Opcua) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol plc4x (PLC4X (Proxy-Protocol)) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol profinet (Profinet) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol s7 (Siemens S7 (Basic)) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering driver for Protocol simulated (Simulated PLC4X Datasource) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Instantiating new PLC Driver Manager with class loader org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader@7ec7ffd3 (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,003] INFO Registering available drivers... (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol ab-eth (Allen Bradley ETH) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol ads (Beckhoff TwinCat ADS) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol bacnet-ip (BACnet/IP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol c-bus (Clipsal C-Bus) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol genericcan (Generic CAN) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol canopen (CAN open) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol eip (EthernetIP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol logix (Logix CIP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol firmata (Firmata) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol iec-60870-5-104 (IEC 60870-5-104) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol knxnet-ip (KNXNet/IP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol open-protocol (Open-Protocol) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol modbus-ascii (Modbus ASCII) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol modbus-rtu (Modbus RTU) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol modbus-tcp (Modbus TCP) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol opcua (Opcua) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol plc4x (PLC4X (Proxy-Protocol)) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,007] INFO Registering driver for Protocol profinet (Profinet) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,008] INFO Registering driver for Protocol s7 (Siemens S7 (Basic)) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,008] INFO Registering driver for Protocol simulated (Simulated PLC4X Datasource) (org.apache.plc4x.java.DefaultPlcDriverManager)
[2025-04-19 13:29:58,008] INFO Assuming job as triggered job because triggerConfig has been set (org.apache.plc4x.java.scraper.config.triggeredscraper.ScraperConfigurationTriggeredImpl)
[2025-04-19 13:29:58,008] INFO Starting jobs... (org.apache.plc4x.java.scraper.triggeredscraper.TriggeredScraperImpl)
[2025-04-19 13:29:58,008] INFO Task TriggeredScraperTask{connectionManager=org.apache.plc4x.java.utils.cache.CachedPlcConnectionManager@7bd0d2d4, jobName='job', connectionAlias='opcserver', connectionString='opcua:tcp://IP_adresse:Port?discovery=false', requestTimeoutMs=2000, executorService=java.util.concurrent.ThreadPoolExecutor@41b95dbb[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], resultHandler=org.apache.plc4x.kafka.Plc4xSourceTask$$Lambda$1106/0x00007f49654e0ce0@1f66168, triggerHandler=org.apache.plc4x.java.scraper.triggeredscraper.triggerhandler.TriggerHandlerImpl@1e702d3a} added to scheduling (org.apache.plc4x.java.scraper.triggeredscraper.TriggeredScraperImpl)
[2025-04-19 13:29:58,011] INFO WorkerSourceTask{id=opc-0} Source task finished initialization and start (org.apache.kafka.connect.runtime.AbstractWorkerSourceTask)
[2025-04-19 13:29:58,017] INFO Configuring Bootstrap with org.apache.plc4x.java.transport.tcp.DefaultTcpTransportConfiguration@2b3401b5 (org.apache.plc4x.java.transport.tcp.TcpChannelFactory)
[2025-04-19 13:29:58,030] INFO Configuring Bootstrap with org.apache.plc4x.java.transport.tcp.DefaultTcpTransportConfiguration@2b3401b5 (org.apache.plc4x.java.transport.tcp.TcpChannelFactory)
[2025-04-19 13:30:00,117] INFO Received open channel response 1, parsing it (org.apache.plc4x.java.opcua.context.SecureChannel)
[2025-04-19 13:30:02,162] INFO Established connection to server (org.apache.plc4x.java.opcua.protocol.OpcuaProtocolLogic)