NSString to unicode (juce_mac_file)

Hi,

I’m trying to use directorycontentlist to scan directories, it works fine … most of the time.

On MAC, if the file name is for exemple “schröder”, I get an unicode (utf16) string like this in my XCode memory browser “s.c.h.r.o…d.e.r.” with length 8 instead of 7.

I think that the “extra” char I’ve got in my string is part of the UTF8 extended representation of chars that can not be encoded on 8 bits.
But this extended char is badly transcoded to utf16.

Does someone has any idea ?

I found a good hint on stack overflow :
http://stackoverflow.com/questions/6153345/different-utf8-encoding-in-filenames-os-x

Maybe the solution
It seems that HFS+ is returning the fully decomposed form of UTF (8 or 16), this seems to be form D (decomposed i suppose). We need to use a composed form.
I’m not a ninja of UTF, so I 'm not sure if you are compatible with the form KC or form C (KC is compatibility composed, and C composed in NSString documentation)

So I just used form C (a guess), you can get if from a NSString with the function precomposedStringWithCanonicalMapping, this returns a NSString * of that form.

basically the fix is on line 1721 of juce_mac_file.mm

and change the code of nsStringToJuce :

return ChatPointer_UTF16 (( const CharPointer_UTF16::CharType * ) [s cStringUsingEncoding:NSUTF16LittleEndianStringEncoding]);

I’m sure it works with the ö … but I let better JUCE and UTF coders confirm this fix.

cheers!

Are you using a very old version of juce? Strings have been utf8 internally for a long long time now.

Hello,

I’m using juce 1.53, but the fact that it is utf8 internally was not the problem it seems, but more the form of the utf8 inHFS+.

In later versions, a utf8 string from the filesystem will be used directly, and never get converted to utf-16 at all, so there can’t be any conversion errors.

If you still have problems with the latest version, let me know, but I can’t spend time looking into bugs that were probably fixed long ago, sorry!

I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?

[quote=“adrien59cadri”]I understand, but there is no later version, except moving to the git trunk.
Do you mean that any fixes will only be on git, and/or with juce quake ?[/quote]

Well, using GIT is the smart way to do it, but even without it you could get 1.54.

And yes, of course all fixes go into the bleeding edge.

I did not even notice that there is a 1.54 version.
I can’t find it neither on sourceforge, neither as a tag on GIT.

Beside that, will the next fixes be only in juce-quake ?

Yes, of course. How else could I do it!? I can’t retro-fit every change I make into new branches of all the previous versions!