NSString to unicode (juce_mac_file)

[adrien59cadri]

Hi,

I’m trying to use directorycontentlist to scan directories, it works fine … most of the time.

On MAC, if the file name is for exemple “schröder”, I get an unicode (utf16) string like this in my XCode memory browser “s.c.h.r.o…d.e.r.” with length 8 instead of 7.

I think that the “extra” char I’ve got in my string is part of the UTF8 extended representation of chars that can not be encoded on 8 bits.
But this extended char is badly transcoded to utf16.

Does someone has any idea ?

I found a good hint on stack overflow :
http://stackoverflow.com/questions/6153345/different-utf8-encoding-in-filenames-os-x

OS X’s HFS+ filesystem requires that all filenames be stored in the UTF-8 representation of their fully decomposed form. In an HFS+ filename, “ä” MUST be encoded as 0x61cc88, and “ö” MUST be encoded as 0x6fcc88.

Maybe the solution
It seems that HFS+ is returning the fully decomposed form of UTF (8 or 16), this seems to be form D (decomposed i suppose). We need to use a composed form.
I’m not a ninja of UTF, so I 'm not sure if you are compatible with the form KC or form C (KC is compatibility composed, and C composed in NSString documentation)

So I just used form C (a guess), you can get if from a NSString with the function precomposedStringWithCanonicalMapping, this returns a NSString * of that form.

basically the fix is on line 1721 of juce_mac_file.mm

and change the code of nsStringToJuce :

return ChatPointer_UTF16 (( const CharPointer_UTF16::CharType * ) [s cStringUsingEncoding:NSUTF16LittleEndianStringEncoding]);

I’m sure it works with the ö … but I let better JUCE and UTF coders confirm this fix.

cheers!

[jules]

I get an unicode (utf16) string like this in my XCode memory browser

Are you using a very old version of juce? Strings have been utf8 internally for a long long time now.

[adrien59cadri]

Hello,

I’m using juce 1.53, but the fact that it is utf8 internally was not the problem it seems, but more the form of the utf8 inHFS+.

[jules]

In later versions, a utf8 string from the filesystem will be used directly, and never get converted to utf-16 at all, so there can’t be any conversion errors.

If you still have problems with the latest version, let me know, but I can’t spend time looking into bugs that were probably fixed long ago, sorry!

[adrien59cadri]

I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?

[jules]

[quote=“adrien59cadri”]I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?[/quote]

Well, using GIT is the smart way to do it, but even without it you could get 1.54.

And yes, of course all fixes go into the bleeding edge.

[adrien59cadri]

I did not even notice that there is a 1.54 version.
I can’t find it neither on sourceforge, neither as a tag on GIT.

Beside that, will the next fixes be only in juce-quake ?

[jules]

Beside that, will the next fixes be only in juce-quake ?

Yes, of course. How else could I do it!? I can’t retro-fit every change I make into new branches of all the previous versions!