I’m trying to use directorycontentlist to scan directories, it works fine … most of the time.
On MAC, if the file name is for exemple “schröder”, I get an unicode (utf16) string like this in my XCode memory browser “s.c.h.r.o…d.e.r.” with length 8 instead of 7.
I think that the “extra” char I’ve got in my string is part of the UTF8 extended representation of chars that can not be encoded on 8 bits.
But this extended char is badly transcoded to utf16.
Does someone has any idea ?
I found a good hint on stack overflow :
OS X’s HFS+ filesystem requires that all filenames be stored in the UTF-8 representation of their fully decomposed form. In an HFS+ filename, “ä” MUST be encoded as 0x61cc88, and “ö” MUST be encoded as 0x6fcc88.
Maybe the solution
It seems that HFS+ is returning the fully decomposed form of UTF (8 or 16), this seems to be form D (decomposed i suppose). We need to use a composed form.
I’m not a ninja of UTF, so I 'm not sure if you are compatible with the form KC or form C (KC is compatibility composed, and C composed in NSString documentation)
So I just used form C (a guess), you can get if from a NSString with the function precomposedStringWithCanonicalMapping, this returns a NSString * of that form.
basically the fix is on line 1721 of juce_mac_file.mm
and change the code of nsStringToJuce :
return ChatPointer_UTF16 (( const CharPointer_UTF16::CharType * ) [s cStringUsingEncoding:NSUTF16LittleEndianStringEncoding]);
I’m sure it works with the ö … but I let better JUCE and UTF coders confirm this fix.
Are you using a very old version of juce? Strings have been utf8 internally for a long long time now.
I’m using juce 1.53, but the fact that it is utf8 internally was not the problem it seems, but more the form of the utf8 inHFS+.
In later versions, a utf8 string from the filesystem will be used directly, and never get converted to utf-16 at all, so there can’t be any conversion errors.
If you still have problems with the latest version, let me know, but I can’t spend time looking into bugs that were probably fixed long ago, sorry!
I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?
[quote=“adrien59cadri”]I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?[/quote]
Well, using GIT is the smart way to do it, but even without it you could get 1.54.
And yes, of course all fixes go into the bleeding edge.
I did not even notice that there is a 1.54 version.
I can’t find it neither on sourceforge, neither as a tag on GIT.
Beside that, will the next fixes be only in juce-quake ?
Yes, of course. How else could I do it!? I can’t retro-fit every change I make into new branches of all the previous versions!