Neuron ESB User Network

The Service Bus for the Connected Business

Message cannot be delivered to subscriber due to firewall issue

Hi,

we ha a setup where a client subscribes to multiple topics. Sometime, due to wrong setup, the client cannot be reached due to firewall issue.

If a message cannot be delivered Neuron retries immediately over and over again without any timeout or TTL on the messages. This causes the CPU to spike and remain constantly above 100%.

The following error is logged:

2017-08-31 14:47:38.250+02:00 [71] ERROR - Unable to connect the client with the topic. Party Id MultipleSubscriber, session Id 4224f99e-676b-4cae-97e8-353cc2294f28, topic Location, callback net.tcp://XXX.XXX.XX:61009/ESBTcpPubSubClient/4224f99e-676b-4cae-97e8-353cc2294f28.
System.ServiceModel.CommunicationException: Unable to communicate with client. Ensure that the client's firewall is not blocking Neuron traffic. Party Id MultipleSubscriber, session Id 4224f99e-676b-4cae-97e8-353cc2294f28, topic Location, url net.tcp://DK-W-K1KEP.kamstrup.dk:61009/ESBTcpPubSubClient/4224f99e-676b-4cae-97e8-353cc2294f28, Exception: Could not connect to net.tcp://XXXXXXXXXX:61009/ESBTcpPubSubClient/4224f99e-676b-4cae-97e8-353cc2294f28. The connection attempt lasted for a time span of 00:00:21.0026596. TCP error code 10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.20.203.10:61009. . ---> System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://XXXXXXX:61009/ESBTcpPubSubClient/4224f99e-676b-4cae-97e8-353cc2294f28. The connection attempt lasted for a time span of 00:00:21.0026596. TCP error code 10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.20.203.10:61009. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 172.20.203.10:61009
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)
at System.ServiceModel.Channels.BufferedConnectionInitiator.Connect(Uri uri, TimeSpan timeout)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at System.ServiceModel.ICommunicationObject.Open()
at Neuron.Esb.Channels.WcfPublishingService`1.CreateClientProxy(TProperties properties, String clientEndpointAddress, String clientIdentity)
at Neuron.Esb.Channels.WcfPublishingService`1.ClientProxyPool.CreateProxy()
at Neuron.Esb.Internal.Pool`1.Take()
at Neuron.Esb.Internal.Pool`1.Use(Action`1 action)
at Neuron.Esb.Internal.FixedSizePool`1.Use(Action`1 action)
at Neuron.Esb.Channels.WcfPublishingService`1.UseProxyConnect(TcpClientInfo client, Action`1 action)
at Neuron.Esb.Channels.WcfPublishingService`1.Connect(TcpClientInfo clientInfo)
--- End of inner exception stack trace ---
at Neuron.Esb.Channels.WcfPublishingService`1.Connect(TcpClientInfo clientInfo)

The obvious solution would be of course to solve the firewall issue, but is there any setting that allow us to discard the message if the client cannot be reached within x number of retries or seconds?

Thank you very much

Fabrizio

Views: 771

Reply to This

Replies to This Discussion

unfortunately, not now.  however, if this is coming from the TCP publishing log file (i.e. Topic), if the publishing service fails to reestablish communication with the client within a minute or so (I can't remember what the exact period is) the client will be removed from teh delivery pool and the topic will not attempt to talk to it again.....so this shouldn't report errors for longer than a minute or so

Hi Marty,

the log is taken from the TCP Publishing service but unfortunately the service retried for several hours, until we noticed the CPU spike and restarted the whole instance.

I'm looking at the log and it goes on from 14:47 to 23:59, and probably the day after as well, so maybe the client is not removed properly?

How would you recover once the client is removed, restarting the service?

Fabrizio

actually...I just looked.  this is at the Connect stage...so it will simply try to connect forever.  its only after connect....if we get blocked delivering a message is where we remove the client..

in the Neuron ESB endpoint health monitor, you can just restart the Topic, rather than restart the entire Neuron ESB runtime

Hi Marty,

we are still experiencing this issue.

Looking at the logs it seems like the client is not removed properly, there's a WARNING saying that the client will be removed and after 15sec it logs an error on the same client.

2017-10-17 00:46:46.418+02:00 [84] WARN - The client could not be contacted after 120 seconds. The client is now expired and will be removed from the connected pool list. Party MultipleSubscriber, Session Id 402ddc9f-918f-4e8a-a393-8ff5292808f6, Topic Container, URL net.tcp://DK-W-T1KEP.kamstrup.dk:61009/ESBTcpPubSubClient/402ddc9f-918f-4e8a-a393-8ff5292808f6


2017-10-17 00:47:07.457+02:00 [112] ERROR - Unable to connect the client with the topic. Party Id MultipleSubscriber, session Id 402ddc9f-918f-4e8a-a393-8ff5292808f6, topic Container, callback net.tcp://DK-W-T1KEP.kamstrup.dk:61009/ESBTcpPubSubClient/402ddc9f-918f-4e8a-a393-8ff5292808f6.
System.ServiceModel.CommunicationException: Unable to communicate with client. Ensure that the client's firewall is not blocking Neuron traffic. Party Id MultipleSubscriber, session Id 402ddc9f-918f-4e8a-a393-8ff5292808f6, topic Container, url net.tcp://DK-W-T1KEP.kamstrup.dk:61009/ESBTcpPubSubClient/402ddc9f-918f-4e8a-a393-8ff5292808f6, Exception: Could not connect to net.tcp://dk-w-t1kep.kamstrup.dk:61009/ESBTcpPubSubClient/402ddc9f-918f-4e8a-a393-8ff5292808f6.

RSS

Neuron ESB Product Support Forums and Communities

Latest Activity

Anupama Nair posted a discussion

Marketo Adapter Invalid Token

Hi,We are using the Marketo adapter to push account updates to Marketo. It works well for some time then starts failing with Invalid Token unless restarted. Is there a configuration that can be done so it can auto refresh the token when required?Thanks!See More
Nov 6, 2023
Sayantini Basak posted a discussion

Maximum payload size(REST API) for requests interfacing to NeuronESB

I am new to Neuron ESB and in our current scenario,We need to process batch transactions comprising of ~1000 records and send them to Neuron ESB for further processing. I would like to understand what is the maximum size of payload that can be transferred using REST interface to Neuron ESB.See More
Jul 22, 2022
Profile IconRobert E Dunie and Sayantini Basak joined Neuron ESB User Network
Apr 28, 2022
Profile IconDayanand, Frederic C, Steffen Greve-Oksfeldt and 1 more joined Neuron ESB User Network
Mar 16, 2022
Profile IconCam Vong and Mitja Luznar joined Neuron ESB User Network
Jan 27, 2022
Profile IconWill Hitzges, Chad Parsons, michael larsen and 4 more joined Neuron ESB User Network
Jun 11, 2021
Anupama Nair posted a discussion

ODBC stored proc polling with temporary tables

We have set up an ODBC adapter to poll a stored proc.We found that if the stored proc has a temporary table defined the rows returned are always 0.Any idea why this would be and what we can do to get around it?See More
Dec 14, 2020
Prasanth Kharade is now a member of Neuron ESB User Network
Dec 30, 2019

Badge

Loading…

© 2024   Created by Neuron Admin.   Powered by

Badges  |  Report an Issue  |  Terms of Service