Neuron ESB User Network

The Service Bus for the Connected Business

I want to implement following scenario using Neuron ESB:

- get big list of working items from somewhere (let's say some database)
- split this list to small "batches" of working items
- send these batches each to particular instance of specific subscriber (Worker), so each that instance will process assigned batch in parallel with other instances
- collect feedback (is processing complete/failed) + implement some fail-over mechanism (resend failed batches, monitor if batches weren't processed during some time interval and etc.)

I started to implement special adapter for Neuron, which I called Dispatcher.
The Dispatcher acts as a subscriber for one Topic (A) and can publish to another Topic (B).

The ides was to get from Topic A:
- list of working items
- requests from Worker to add/remove them into/from worker list

Then, when Dispatcher gets a list of working items, then split that list into batches, and then publish each batch to Workers using some load-balancing algorithm (let's say round-robin).

Unfortunately I've got following problems:

- I did not find how to identify an instance of Worker subscriber. The DestId property of ESBMessage works only when I put Party identifier in it. So, as in my case, when each Worker is actually an instance of the same Party, all Workers will get the message (I used TCP Topic);

- Modification of esbService.exe.config is required if you want to send messages under specific part from Neuron Adapter ( have to be added, otherwise Party.Connect() method will fail).
Does anybody tried to implement load-balancing in Neuron ESB (not using MSMQ)?
Or, do you know how to by-pass problems I have?

Tags: balance, dispatching, distribution, load

Views: 425

Reply to This

Replies to This Discussion

Hi Stas,

I think we worked with all out with Sai. You can actually get the list of subscribers within a Process attached to the publishing party. Then you can simply do Send() via Direct to specific parties using the Publish step. no need for the complexity you describe above :)

Kind regards,

Marty

Hi Marty,

Thank you for reply.

Registration/Enumaration of parties is not a problem. Main problem here exactly in "do Send() via Direct to specific parties". I did not find how to identify an instance of a party. When I send message via Direct all parties, listening on the Topic get a message.

Do you have any working example?

Thank you,

Stas

RSS

Neuron ESB Product Support Forums and Communities

Latest Activity

Anupama Nair posted a discussion

Marketo Adapter Invalid Token

Hi,We are using the Marketo adapter to push account updates to Marketo. It works well for some time then starts failing with Invalid Token unless restarted. Is there a configuration that can be done so it can auto refresh the token when required?Thanks!See More
Nov 6, 2023
Sayantini Basak posted a discussion

Maximum payload size(REST API) for requests interfacing to NeuronESB

I am new to Neuron ESB and in our current scenario,We need to process batch transactions comprising of ~1000 records and send them to Neuron ESB for further processing. I would like to understand what is the maximum size of payload that can be transferred using REST interface to Neuron ESB.See More
Jul 22, 2022
Profile IconRobert E Dunie and Sayantini Basak joined Neuron ESB User Network
Apr 28, 2022
Profile IconDayanand, Frederic C, Steffen Greve-Oksfeldt and 1 more joined Neuron ESB User Network
Mar 16, 2022
Profile IconCam Vong and Mitja Luznar joined Neuron ESB User Network
Jan 27, 2022
Profile IconWill Hitzges, Chad Parsons, michael larsen and 4 more joined Neuron ESB User Network
Jun 11, 2021
Anupama Nair posted a discussion

ODBC stored proc polling with temporary tables

We have set up an ODBC adapter to poll a stored proc.We found that if the stored proc has a temporary table defined the rows returned are always 0.Any idea why this would be and what we can do to get around it?See More
Dec 14, 2020
Prasanth Kharade is now a member of Neuron ESB User Network
Dec 30, 2019

Badge

Loading…

© 2024   Created by Neuron Admin.   Powered by

Badges  |  Report an Issue  |  Terms of Service