Neuron ESB User Network

The Service Bus for the Connected Business

Demystifying Durable and Transactional enterprise deployments

Many times when on site I'm asked how do we achieve "guaranteed delivery" or "zero message loss"? with Neuron. The answer is somewhat involved but hopefully after reading this blog you'll understand the pieces that need to be in place.

First, is the environment 2003/XP? If it is, then the story is that if all messaging i.e. parties will run on the server then you run one Neuron server and use the MSMQ channel. Cluster the Neuron box for HA.

The only scaling that is available in this configuration by buying a bigger box...

If you have clients i.e. remote parties that communicate with the server in a 2003/XP environment then you must also use Neuron's server cache mode to prevent message loss.

If the environment is 2008/Vista then you gain some flexibility. You can now move the MSMQ server to a separate box. You'll want to cluster it for HA. Then you can point multiple Neuron servers using the MSMQ channel at the MSMQ box for scaling. MSMQ 4.0 supports remote transactional reads so you do not use server cache mode in this configuration.

One of the hot debates within the Neuron team is should we provide another durable and transactional topic channel say based on a database so that we could enable more flexible farm scenarios...

If you think this would be valuable, I'd appreciate comments or go ahead and start a thread on the forum!


Views: 160


You need to be a member of Neuron ESB User Network to add comments!

Join Neuron ESB User Network

Comment by Brandon on February 11, 2009 at 12:31pm
The Msmq adapter will no longer turn off due to errors starting from build 2.0.1176.0025
Comment by CM on February 10, 2009 at 6:37am
Hi Frosty,

As I indicated in the post, for remote Neuron clients you need the MSMQ channel + server cache mode when using MSMQ 3.0. Using this combination in conjunction with a local persistent store can be used to develop a store and forward scenario with Neuron.

However, the Party classes (Subscriber, Publisher) were not designed for use in IIS. They are doing a lot in the background in order to receive messages on Topics and also to download their pipelines and keep them up to date.

In this case, the choice of a service that monitors the queues and uses a transactional write to a queue on the Neuron server which would then pick up the message via the MSMQ adapter is better or reading the message from the local persistent store in the service and forwarding it via the Neuron API works as well.

We will definitely look into the MSMQ Adapter behavior you're describing,

Finally, thanks for posting the first comment in history!

Comment by Frosty on February 9, 2009 at 9:11pm
Guaranteed deliver / zero message loss can indeed be important, and in our environment is one that we depend on. Your above scenario hold true, given that there has been no network outage between the remote client and the Neuron server. In our scenario, which is primarily integration based, we must still guarantee message delivery (even in a very limited disconnected state), This message pattern is one way (an event) and needs to be as undisruptive as possible to the application generating the event. Our solution was to use a store and forward pattern, initially writing the message to a local (to the publishing application) MSMQ and then asynchronously sending the message to Neuron either in process or using an out of process service to send then message.

When using the in process solution, we have found that Neuron (or more specifically the Neuron API) appears to hang on to resources (maybe due to garbage collection or something in IIS... we have not been able to completely determine that exact cause) and does not allow us to immediately deploy an update to the publishing application, even when restarting the application pool. Restarting IIS is not an option.

This leads us to solution B, out of process sending messages using the Neuron API. We monitor the local queue(s) and when a message is received, the service application will dequeue the message and forward on to Neuron. This has its own set of flaws, but appears to make the architect of this application happy. I see more problems down the road, but for now this works.

We originally thought we could solve this using an MSMQ adapter, but alas, if the Neuron server and the remote machine hosting the publishing application are disconnected for more than a couple of minutes, then the Neuron MSMQ adapter gives up and will not try to connect again unless the ESB service is restarted. We may revisit this strategy and write our own version of an MSMQ adapter that degrades a little more gracefully.

Did I miss an obvious solution or given our requirement of 100% (not 99.5%) guaranteed message (event) delivery, did I take a reasonable approach?

Neuron ESB Product Support Forums and Communities

Latest Activity

Anupama Nair posted a discussion

Marketo Adapter Invalid Token

Hi,We are using the Marketo adapter to push account updates to Marketo. It works well for some time then starts failing with Invalid Token unless restarted. Is there a configuration that can be done so it can auto refresh the token when required?Thanks!See More
Nov 6, 2023
Sayantini Basak posted a discussion

Maximum payload size(REST API) for requests interfacing to NeuronESB

I am new to Neuron ESB and in our current scenario,We need to process batch transactions comprising of ~1000 records and send them to Neuron ESB for further processing. I would like to understand what is the maximum size of payload that can be transferred using REST interface to Neuron ESB.See More
Jul 22, 2022
Profile IconRobert E Dunie and Sayantini Basak joined Neuron ESB User Network
Apr 28, 2022
Profile IconDayanand, Frederic C, Steffen Greve-Oksfeldt and 1 more joined Neuron ESB User Network
Mar 16, 2022
Profile IconCam Vong and Mitja Luznar joined Neuron ESB User Network
Jan 27, 2022
Profile IconWill Hitzges, Chad Parsons, michael larsen and 4 more joined Neuron ESB User Network
Jun 11, 2021
Anupama Nair posted a discussion

ODBC stored proc polling with temporary tables

We have set up an ODBC adapter to poll a stored proc.We found that if the stored proc has a temporary table defined the rows returned are always 0.Any idea why this would be and what we can do to get around it?See More
Dec 14, 2020
Prasanth Kharade is now a member of Neuron ESB User Network
Dec 30, 2019



© 2024   Created by Neuron Admin.   Powered by

Badges  |  Report an Issue  |  Terms of Service