DirectoryIterator possibly broken in High Sierra (Solved... not JUCE bug)


#1

It appears that DirectoryIterator is malfunctioning with High Sierra’s new file system.

Our plugins’ Preset Menu loads an alphabetized list of presets using this function:

void UAPresetMenu::presetFolderConstructor(File &fileToSearch, PopupMenu &menuToPopulate, int &fileIndex)
{
    DirectoryIterator userIterator(fileToSearch, false, "*", File::findFilesAndDirectories);

    while (userIterator.next())
    {
        File nextFile = userIterator.getFile();
        if (nextFile.isDirectory())
        {
            PopupMenu presetFolder;
            presetFolderConstructor(nextFile, presetFolder, fileIndex);
			String name = nextFile.getFileName();
            menuToPopulate.addSubMenu(name, presetFolder);
        }
        else if (nextFile.hasFileExtension(".uap"))
        {
            menuToPopulate.addItem(fileIndex, nextFile.getFileNameWithoutExtension());
            presetList.push_back(nextFile);
            fileIndex++;
        }
    }
}

This code works correctly on WinXP through Win 10 and Snow Leopard through Sierra. High Sierra is the first to misbehave. Here’s an example of the seemingly random order:


#2

issue confirmed here under 10.13


#3

I didn’t see anywhere in the documentation for the class that it claims the order searched will be alphabetic. Did I miss something? https://www.juce.com/doc/classDirectoryIterator#details


#4

It’s not supposed to return sorted items, no. It returns them in the same order as the underlying file-system returns them. You’ll have to sort your returned items yourself.


#5

ah, that’s right! I guess we were just expecting that because it has always been the default behaviour.
Anyway, that should do :

DirectoryIterator iter (directory, false, "*", File::findFilesAndDirectories);

Array<File> files;

while (iter.next())
    files.addUsingDefaultSort (iter.getFile());

for (auto& f : files)
{
}

#6

Nice! I was trying to roll a solution over here but this is much cleaner. Many thanks.

I did look into solving it at a lower level near the “NSDirectoryEnumerator,” however I wasn’t finding an easy way to make that happen. Everything I found suggested a manual sort at that level which would be cool but then it would be hard to pass up the chain without rewriting a bunch of internal JUCE since all of those functions are expecting an enumerator instead of a sorted array. For example: https://stackoverflow.com/questions/19716459/sorting-of-array-returned-by-a-nsfilemanagers-subpathsatpath-method


#7

Yes, the DirectoryIterator definitely gives no guarantee about the order - I’m amazed that you were ever lucky enough to see them in alphabetical order at all!

A slightly better sort might be like the one we use inside Projucer:

struct FileSorter
{
    static int compareElements (const File& f1, const File& f2)
    {
        return f1.getFileName().compareNatural (f2.getFileName());
    }
};

Array<File> files;
getFolder().findChildFiles (files, File::findFiles, false);

FileSorter sorter;
files.sort (sorter);

The compareNatural method is smarter than a simple string comparison, and does a better job of sorting numeric prefixes and other things


#8

and juce already has a file sorter ! :slight_smile:

Array<File> files;
getFolder().findChildFiles (files, File::findFilesAndDirectories, false);

bool putFoldersFirst = false;
File::NaturalFileComparator sorter (putFoldersFirst);
files.sort (sorter);

#9

Ah, thanks, I forgot about that one!


#10

Thanks, everyone!


#11

@jules, imho it’s a really common operation to get a files list in natural order.
I already see I need it in different projects, and the projucer or KnownPluginList could also make use of it I guess.

adding an extra optional sortNatural parameter (perhaps even true by default) to File::findChildFiles() could save some typing.
I’m pretty sure many of us would use it (e.g. when building menus listing presets files).
What’s your opinion? do you think it’s to much specific or would you consider adding it?
patch.txt (1,5 Ko)


#12

Not really keen on adding yet another parameter to that method, which already has too many!

And sorting is something that would lead to feature-creep, I can imagine that if we had that parameter, next thing would be “hey can you just add another option to ignore leading numbers?”, “can you add an option to make it ignore case?”, “can you add an option to put folders first?” etc etc

Much better to keep those operations and decisions away from the File class.