If you write to a file (using File.replaceWithText() or such) with asUnicode set to true and then read from the file later, JUCE bugs out.
When reading from the affected file in any manner (File.loadFileAsString, File.readLines, FileInputStream.readEntireStreamAsString, etc.), only the first character is returned.
The only way I have found to get the whole file is a loop using FileInputStream.readNextLine, but this returns the characters one by one instead of lines.
Is there a different way read from files that supports unicode?
The name of the asUnicode parameter is misleading, it will actually cause the file to be written as UTF-16. If you omit the byte-order mark by setting the third parameter to ‘false’, then loadFileAsString() will have no way of determining the correct text encoding of the file, and will assume UTF-8. I recommend ensuring that the asUnicode and writeHeaderBytes parameters match. This should produce the expected results.
I see, thanks! But is there a reason that asUnicode and writeHeaderBytes are separate parameters? I can’t think of a use-case that would need one but not the other.
In the case of appendText(), the file might already exist, in which case it’s reasonable to want to write extra UTF-16 to the end of the file without including the header bytes (byte-order mark).
I’m not sure about replaceWithText(), but I can imagine situations where a JUCE program must write a UTF-16 file that doesn’t include a byte-order mark, perhaps to interoperate with another program that always assumes a UTF-16 encoding for input files.
