Neuron ESB User Network

The Service Bus for the Connected Business

I have a restful web service that accepts an XML payload via POST and processes it.  The intent is for batch processing of records.  It is currently configured as a Service Endpoint in the ESB.  Because it is restful, my only message pattern choice is Request/Reply.

Is it possible to configure a restful endpoint to be multicast?  Processes that access this endpoint will not care too much about the responses, however this is a situation where a process will fetch data then publish multiple messages to this endpoint.  I'd like to not have to wait for each message to be responded to before moving onto the next- the results are not contiguous.

Because of the size of the initial payload, the message on the ESB bus is only a pointer to a file.  I do not have the option to use a parallel split to get around the request-reply nature and make multiple requests at once.

Are there other options that I'm missing?

Views: 390

Reply to This

Replies to This Discussion

You cannot configure a REST Service Endpoint as multicast, since by definition all REST calls require some type of response from the service (even if it is an instant 200 OK).  There are a few patterns you could implement in Neuron to quickly return a response.  Based on your description, it almost sounds like you have a Neuron process calling a REST service endpoint hosted by Neuron?



The endpoint isn't hosted by Neuron.  It's a WebAPI service.  Here's the flow:

  • A message is published to the topic
  • The on publish process related to that publisher splits the message into many smaller messages
  • Each smaller messages is then published another topic, the subscriber of which is this service endpoint
  • WebAPI service receives (because the service endpoint passed on the message) and performs the correct task with the smaller message

I was looking for a way to have this process run in parallel- each smaller message is independent of the other.

Have you tried the split step in a process?  You can run this in async mode if ordering doesn't matter.  Also, instead of publishing to a topic for the service endpoint to subscribe to, you should use the Service Endpoint step. Both of these will speed-up your process.

Those are great, but the content of the ESB message isn't the actual data.  It's a small bit of XML saying "find the real data in this file".  That's what I meant by "pointer to a file".

Because the message content isn't the actual data, there's nothing for a Split step to actually split.  I am using a file stream to process the data into the smaller messages, but its happening inside a C# step.

I can build a threadpool and have that C# step do this work through a webclient-  I was merely looking to see if I could utilize Neuron to perform that function in order to capitalize on its built-in controls for things like bulk service calls.  In order to facilitate that goal I ended up instantiating a publisher inside C# that would publish the smaller messages, hoping to gain parallelism by firing off lots of smaller messages and having the system process them individually.  Like a scatter with no gather, but I need to use C# to kick it off.

This sounds like a workflow problem.  You can have the REST client connector publish to a topic that is subscribed to by a workflow endpoint.  The workflow endpoint can take the mini-message and pull the data and do whatever you want with it.  The client connectors publisher will still have a process executed on the on-publish event,  The basic pieces of that process would be:

1.  publish step to durable topic that workflow endpoint subscribes to, semantic = multicast

2.  c# step to step http response code to 200

3.  Cancel step to force return of service call to client

Workflows can be configured to have as many as you want running concurrently.  Any messages that are waiting to be processed will be durably stored.


Neuron ESB Product Support Forums and Communities

Latest Activity

Anupama Nair posted a discussion

Marketo Adapter Invalid Token

Hi,We are using the Marketo adapter to push account updates to Marketo. It works well for some time then starts failing with Invalid Token unless restarted. Is there a configuration that can be done so it can auto refresh the token when required?Thanks!See More
Nov 6, 2023
Sayantini Basak posted a discussion

Maximum payload size(REST API) for requests interfacing to NeuronESB

I am new to Neuron ESB and in our current scenario,We need to process batch transactions comprising of ~1000 records and send them to Neuron ESB for further processing. I would like to understand what is the maximum size of payload that can be transferred using REST interface to Neuron ESB.See More
Jul 22, 2022
Profile IconRobert E Dunie and Sayantini Basak joined Neuron ESB User Network
Apr 28, 2022
Profile IconDayanand, Frederic C, Steffen Greve-Oksfeldt and 1 more joined Neuron ESB User Network
Mar 16, 2022
Profile IconCam Vong and Mitja Luznar joined Neuron ESB User Network
Jan 27, 2022
Profile IconWill Hitzges, Chad Parsons, michael larsen and 4 more joined Neuron ESB User Network
Jun 11, 2021
Anupama Nair posted a discussion

ODBC stored proc polling with temporary tables

We have set up an ODBC adapter to poll a stored proc.We found that if the stored proc has a temporary table defined the rows returned are always 0.Any idea why this would be and what we can do to get around it?See More
Dec 14, 2020
Prasanth Kharade is now a member of Neuron ESB User Network
Dec 30, 2019



© 2024   Created by Neuron Admin.   Powered by

Badges  |  Report an Issue  |  Terms of Service