JavaScript and accented characters


#1

Hi,

We're working on an application that makes recommendations to adjust chemical levels based on values read from a device. Because the rules can potentially be more involved than simple if/else, we decided to use scripting. Just before I went off down the path of integrating Lua or Python, I noticed that JUCE has a JavaScript interpreter built in. Brilliant.

However, the recommendations need to be in multiple languages, and while I've gotten everything how I like it with the English version, the parser seems to fall flat if there are any accented characters in the JavaScript source. Is this a known deficiency, or is there a workaround? I'd have thought that as UTF and wide strings are extensively supported elsewhere in the framework, this wouldn't be an issue with the interpreter. Additionally the JUCER eliminates any non-ASCII characters (my French script doesn't display in the JUCER at all, and when I manually added a French accented word in the English script, half of the characters were removed).

Am I looking at having to implement an alternative scripting approach...?

 


#2

There's no reason why the javascript parser would care what characters are involved, and inside JUCE all strings are unrestricted UTF8.. What exactly is failing to work? Got an example?

(We get lots of posts like this where people say "unicode doesn't work", but in 99% of cases it's because because people misunderstand how compilers treat literals inside source code, and they're saving their files with local encodings, but loading them as if they're UTF8)


#3

Hi Jules,

The file is edited in Visual Studio, but when it's saved it appears perfectly fine (all the funky French characters) when I view it in Sublime Text.

We're temporarily adding the file to our binaries for simple access, but it's going to be stored in SQL later - when I try to view the file in JUCER it just displays the filename; the English version actually displays the entire source file. If I add French characters to the English file, those characters are eliminated when viewing in JUCER.

It's a proprietary product so I can't post the script (we've licensed JUCE, btw, nothing dodgy!), but I can send you a failing script if you like. The error returned indicates it's failed to find the closing of a string, but the line and column don't correspond...

 


#4

The file is edited in Visual Studio, but when it's saved it appears perfectly fine (all the funky French characters) when I view it in Sublime Text.

Sure, but maybe Sublime Text sees that the file isn't valid UTF8 and so displaying it using a local code-page?

The juce stuff won't do that - it expects to be given valid UTF8, and I'm not convinced from your comments that you really understand the difference between that and a local encoding! You're almost certainly just feeding it non-UTF8 data.