Directory Iterator. Time to scan

Useful Tools and Components
1

Hi,

I just found a weird behaviour with the Directory Iterator on my PC.

I scan big directories with a "*.wav" wildcard. The time taken to do this scan can change a lot. It usually takes around 3 seconds, but sometimes it takes up to 60 seconds, for the same result.

How can it sometimes take that long ? 

My directory iterator is a local object, in an independant thread.

After some research, I found out that the next() instruction of the directory iterator object can sometimes take very long when the file is far away from the previous one in the file tree.

Any idea ?

 

2

I find this to be true when accessing big directories through Explorer. And it seems to be exacerbated when I hit one of my external USB drives. My anecdotal explanation is both caching and (in the case of the USB) possibly waking up. I'd be interested to know if it is something else in your case.

3

Hi,

Thanks for these details. 

In my case, I don't scan USB drives.

It seems likely that this is OS caching related, as the scan seems to take long only when I choose a new directory.

Jules, can you confirm that?

 

4

Yep. All the juce classes do is to call the OS functions to look for files - if there's a delay it'll be because the OS is slow in returning them.

5

On Linux the entire available system RAM is used for disk caching (which is why it can appear as if the system never has any free ram) and the difference between cached & non-cached directory scans is huge, even when reading off a super fast SSDs.

Is the main point of DirectoryContentList's own thread to just prevent the current thread from locking up while waiting for files to load?

thx & cheers,

-- p