FTP adapter issue with funny characters

Replies to This Discussion

Permalink Reply by FabrizioM. on February 19, 2018 at 6:32am

Hi Per,

it has something to do with the encoding of the message.

I had the same issue with danish characters and when the message does not specify UTF-8 encoding that's how it looks like in the Message History.

Not really sure how to solve it in your scenario but maybe it can help.

Fabrizio

Permalink Reply by Martin Wasznicky on February 19, 2018 at 11:39am

Ok…I looked into this.

 

This is behaving as designed though…but I think we can make a tweak that makes sense.

 

In short the data is being encoded as ASCII because the file with  the data on the FTP server is saved as an ASCII file.  If this file was saved as a Unicode file on the FTP server, the data would come across as you would expect.  Internally, once we receive the data (regardless of the way we download i.e. ASCII or binary) we look at the BOM of the file to try to determine the encoding.  If the file BOM tells us its ASCII, then we use that encoder.

 

Solution 1.) What you could do to correct this is ensure that the file is saved as a Unicode file on the FTP server.  That way, we’ll detect the correct encoding.

 

Solution 2.) The tweak I was talking about that we could make though is to ensure that we simply just pass the data through as binary, if Binary is selected as the Transfer type…..which kind of makes sense.

 

But that means you’ll get a byte array as the esb message body.  You’ll have to do use the Unicode encoder within a Business Process to decode it to the format that you will expect….providing you need to work with the data in text format. 

 

Thoughts?

Permalink Reply by Martin Wasznicky on February 19, 2018 at 2:34pm

Actually....took this a step further....added a "Detect Encoding" property....default value is true.  When true....existing functionality would stay the same regardless of transfer type setting (i.e. Binary or ascii).  however, if set to false, we would pass the data through as a byte array....

 

then in a process, you could do something like this in a C# step:

var value = System.Text.Encoding.GetEncoding("utf-7").GetString(context.Data.Bytes);  

context.Data.FromString(value);

Permalink Reply by Per Laustsen on February 20, 2018 at 11:31pm

Thanks for your reply, I appreciate it.

However, it seems I'm still not able to get the correct output.

Could it be that Neuron alters the file input, i.e. adds the replacement character � where it finds the characters æ,ø,å?

I have tried your solution with several different encodings, and also added following:

context.Data.Header.Binary = true;

Otherwise the C# step could not be validated.

Permalink Reply by Martin Wasznicky on February 21, 2018 at 8:01am

Hi Per,

 

it will ONLY work with the updated adapter.  Remember, in my last reply I said I added the "detect encoding" property.  if you set it to false, then you can use the c# step code sample to decode....

 

I'll send you a link to the updated adapter

Permalink Reply by Per Laustsen on February 22, 2018 at 1:02am

Ok thanks Marty, it works.

Thanks for the quick response! :)