Xml getDoubleAttribute issue


#1

I’m having an issue reading doubles from an xml file. I’ve simplified the code and posted it below. Am I doing something wrong here? (I’m using JUCE 5.1.,0, Ubuntu 16.04)

String xmlData = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
"\n"
"<FILTER uid=\"1\" x=\"0.4257309941520467933\" y=\"0.14153846153846152967\"\n"
"        uiLastX=\"0\" uiLastY=\"0\">\n"
"</FILTER>";    

ScopedPointer<XmlElement> xml = XmlDocument::parse(xmlData);
//prints 0 instead of 0.4257309941520467933 
Logger::writeToLog( String(xml->getDoubleAttribute ("x")));

#2

Seems to be Linux specific, here on Mac it works as expected:

So I guess, no you don’t do something wrong…


#3

I compiled and ran that code on Ubuntu 14.04 with JUCE 5.1.0, and I got

JUCE v5.1.0
0.425731

#4

That’s also working correctly for me on both MacOS and Linux. Can you try using the latest develop version?


#5

Thanks everyone. I’ll try the latest dev branch a little later.


#6

Are you perhaps running this on a computer where the language settings specify the comma “,” as the decimal separator instead of the dot “.” that’s commonly used in English?

If that’s the case, it’s possible that the parsing routine is stopping at the dot because it does not recognize it as a valid character in a double number, and returns the truncated value.

If that turns out to be the problem, I think that JUCE should make sure to write and read numbers always in the same format (probably with the dot) regardless of locale settings, because that could be a problem when exchanging XML between computers with different localizations.

I remember that was an issue at the times of JUCE 1.xx, but I think that was fixed somehow in following versions (at least for Mac and Windows, which are the platforms that I build).
Maybe this is still an issue on Linux, I have never built for that platform.

@tom perhaps adding a unit test for this could be useful (simply check that a certain piece of XML parses to the correct decimal value), just in case to avoid regressions?


#7

Now that you mention it, I have been in Germany for the past month, where they use the comma. Perhaps my Linux distro change some regional settings. I still haven’t got around to updating, but I’m back I’ll be back in Ireland today, so the problem may resolve itself…


#8

Making it one of the hardest to track down, because it depends on the country where you do your development :smile:

No seriously, if that turns out to be the problem, then I believe some work will be needed to guarantee that floats are always emitted and parsed with the same decimal separator, and some unit test code to ensure that it happens at all times is certainly welcome to prevent regressions, for example when adding supported platforms


#9

I no way, any regional setting, should have any influence how a xml is parsed! We are not taking about an UI-element, where this might be okay, but for internal data processing, imho this would be horrible.


#10

I don’t think this is the issue though. getDoubleAttribute will eventually call CharacterFunctions::readDoubleValue which has the decimal point hard-coded as '.'. So, I’m really not sure what’s going on here.


#11

When I first discovered the issue I stepped through the debugger and saw that String::getDoubleValue() was not returning anything after the decimal place. When I read the attritube as a string it prints the full number. But I also tried using atof on the string and that also failed. Anyhow, I will try update later and see if it is resolved.


#12

atof definitely DOES use the current locale, so it may very well misinterpret the dot if the current locale uses the comma.


#13

Ah, I recently made a change so that we use strtod to parse doubles, which uses the current locale :frowning: . I’ll fix this.


#14

:wink:


#15

Do you mind, adding that as an option, for people writing localised software? Not for writing XML obviously, but the method is very generic, I guess…


#16

Maybe that could be made as a boolean parameter in the parsing/emitting functions?

Certainly to be specified as “don’t use locale” when parsing or emitting XML, but in certain other contexts that may make sense. Like the ignoreCase parameter seen in some other String methods


#17

There are already unit tests that would catch this, but we don’t run the tests on a machine with a different locale. I’ve just configured a new Ubuntu VM to use German and I still can’t reproduce the problem…

I’m convinced strtod is the culprit though.


#18

Just in case this is not what you are doing but if you test just write a double then read it back you won’t see the issue.
You need your test to use predefined string with dot and comma.


#19

We parse strings containing dots:

This will fail on a system expecting commas.


#20

I would add comma in your test just to be sure :slight_smile: