Neuron ESB User Network

The Service Bus for the Connected Business

Demystifying Durable and Transactional enterprise deployments

Many times when on site I'm asked how do we achieve "guaranteed delivery" or "zero message loss"? with Neuron. The answer is somewhat involved but hopefully after reading this blog you'll understand the pieces that need to be in place.

First, is the environment 2003/XP? If it is, then the story is that if all messaging i.e. parties will run on the server then you run one Neuron server and use the MSMQ channel. Cluster the Neuron box for HA.

The only scaling that is available in this configuration by buying a bigger box...

If you have clients i.e. remote parties that communicate with the server in a 2003/XP environment then you must also use Neuron's server cache mode to prevent message loss.

If the environment is 2008/Vista then you gain some flexibility. You can now move the MSMQ server to a separate box. You'll want to cluster it for HA. Then you can point multiple Neuron servers using the MSMQ channel at the MSMQ box for scaling. MSMQ 4.0 supports remote transactional reads so you do not use server cache mode in this configuration.

One of the hot debates within the Neuron team is should we provide another durable and transactional topic channel say based on a database so that we could enable more flexible farm scenarios...

If you think this would be valuable, I'd appreciate comments or go ahead and start a thread on the forum!




--CM

Views: 148

Comment

You need to be a member of Neuron ESB User Network to add comments!

Join Neuron ESB User Network

Comment by Brandon on February 11, 2009 at 12:31pm
The Msmq adapter will no longer turn off due to errors starting from build 2.0.1176.0025
Comment by CM on February 10, 2009 at 6:37am
Hi Frosty,


As I indicated in the post, for remote Neuron clients you need the MSMQ channel + server cache mode when using MSMQ 3.0. Using this combination in conjunction with a local persistent store can be used to develop a store and forward scenario with Neuron.


However, the Party classes (Subscriber, Publisher) were not designed for use in IIS. They are doing a lot in the background in order to receive messages on Topics and also to download their pipelines and keep them up to date.

In this case, the choice of a service that monitors the queues and uses a transactional write to a queue on the Neuron server which would then pick up the message via the MSMQ adapter is better or reading the message from the local persistent store in the service and forwarding it via the Neuron API works as well.

We will definitely look into the MSMQ Adapter behavior you're describing,

Finally, thanks for posting the first comment in neuronesb.com history!

--CM
Comment by Frosty on February 9, 2009 at 9:11pm
Guaranteed deliver / zero message loss can indeed be important, and in our environment is one that we depend on. Your above scenario hold true, given that there has been no network outage between the remote client and the Neuron server. In our scenario, which is primarily integration based, we must still guarantee message delivery (even in a very limited disconnected state), This message pattern is one way (an event) and needs to be as undisruptive as possible to the application generating the event. Our solution was to use a store and forward pattern, initially writing the message to a local (to the publishing application) MSMQ and then asynchronously sending the message to Neuron either in process or using an out of process service to send then message.

When using the in process solution, we have found that Neuron (or more specifically the Neuron API) appears to hang on to resources (maybe due to garbage collection or something in IIS... we have not been able to completely determine that exact cause) and does not allow us to immediately deploy an update to the publishing application, even when restarting the application pool. Restarting IIS is not an option.

This leads us to solution B, out of process sending messages using the Neuron API. We monitor the local queue(s) and when a message is received, the service application will dequeue the message and forward on to Neuron. This has its own set of flaws, but appears to make the architect of this application happy. I see more problems down the road, but for now this works.

We originally thought we could solve this using an MSMQ adapter, but alas, if the Neuron server and the remote machine hosting the publishing application are disconnected for more than a couple of minutes, then the Neuron MSMQ adapter gives up and will not try to connect again unless the ESB service is restarted. We may revisit this strategy and write our own version of an MSMQ adapter that degrades a little more gracefully.

Did I miss an obvious solution or given our requirement of 100% (not 99.5%) guaranteed message (event) delivery, did I take a reasonable approach?

Neuron ESB Product Support Forums and Communities

Latest Activity

Prasanth Kharade is now a member of Neuron ESB User Network
Dec 30, 2019
Suman vadde is now a member of Neuron ESB User Network
Dec 16, 2019
Morten Andreasen is now a member of Neuron ESB User Network
Dec 9, 2019
Profile IconCarl Porch, Cordier, Lahbib Marouan and 4 more joined Neuron ESB User Network
Nov 18, 2019
Miroslav Jelev posted a discussion

Using unsupported OAuth provider

Is it possible to create a custom OAuth provider?  Specifically, we are working with Cisco (https://cloudsso.cisco.com/as/token.oauth2).We have a requirement to integrate with a Cisco soap server which uses OAuth2 for authentication. We are using a Service Connector to send a message to the SOAP service, but we need to write custom code in a business process to authenticate with their Cisco OAuth provider. We saw that Neuron supports…See More
Jul 29, 2019
Profile Iconchris comer, Fahd EL YOUBI, Vishal Misal and 2 more joined Neuron ESB User Network
May 16, 2019
Alixx Skevington posted a discussion

Docker Windows 2019 - Neuron 3.5

Hi, We are using 3.5 for our application and I am trying to containerise it for our integration tests.  Building and deploying VM's as this is very time consuming to build and deploy.So I have decided that I to go down docker; everything in our stack has migrated nicely except Neuron.  The first issue was that I needed a later version of the 2016 image or 2019 image from MS which I have. I have managed to install MSMQ on it as well. But when I try to run a silent install everything runs.  But…See More
Apr 15, 2019

Badge

Loading…

© 2020   Created by Neuron Admin.   Powered by

Badges  |  Report an Issue  |  Terms of Service