Long path names in windows breaks juce API in multiple places

If you are dealing with long path names (exceeding 260 chars) on windows 10, even if you enabled long path names via Registry or Local Group Policy Editor (depending on the version of your windows), juce fails creating or accessing files, an example:

void FileOutputStream::openHandle()
{
    auto h = CreateFile (file.getFullPathName().toWideCharPointer(),
                         GENERIC_WRITE, FILE_SHARE_READ, nullptr,
                         OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);

    if (h != INVALID_HANDLE_VALUE)

fails badly with long filenames. We have some other CreateFile calls, and this could potentially affect other calls like CreateNamedPipe, SetFileAttributes, RemoveDirectory, DeleteFile and in general every windows API function that takes a path.

To check a possible fix, i was able to deal with those files if i mangled the path to contain the L"\\\\?\\" prefix and ensuring the unicode version of the API is used as in:

void FileOutputStream::openHandle()
{
    auto h = CreateFileW ((juce::String(L"\\\\?\\") + file.getFullPathName()).toWideCharPointer(),
                         GENERIC_WRITE, FILE_SHARE_READ, nullptr,
                         OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);

    if (h != INVALID_HANDLE_VALUE)

I also found a similar unanswered 4 years old topic about the issue: Windows MAX_PATH

Using shorter paths is not an option, and will not be considered an accepted solution.

1 Like

For example, as stated in the https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew they report:

Tip Starting with Windows 10, version 1607, for the unicode version of this
function (CreateFileW), you can opt-in to remove the MAX_PATH limitation
without prepending "\\?\". See the "Maximum Path Length Limitation"
section of Naming Files, Paths, and Namespaces for details.

so we could detect the version of windows we run on, if it has long paths enabled so to prepend or not the prefix.

You are almost there: instead of of hardcoding this you could add:

    juce::String longFilenamePrefix(juce::SystemStats::getOperatingSystemType() >= juce::SystemStats::Windows10 ? L"\\\\?\\" : L"");

Then add this longFilenamePrefix where it is needed and you could propose a patch so that it won’t break other OS’es ?

PS: I assume here that version is >=1607 which could be very acceptable for windows users that usually keep updating their OS and virus definitions?
(i.e. I run version 1909 ATM according to the “winver” tool)
It would be better if we could read the version too, but could not find the juce API for this.
EDIT: the build number for windows 10 1607 would be 14393.
If we could read that build number, we could check that this number should also be >= 14393…
The build number can be read in the registry:
Under:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion
which has values for CurrentMajorVersionNumber, CurrentMinorVersionNumber, and CurrentBuildNumber.

I’ve tried but i still had to add the prefix there, so not a solution. I have a PR for this but i’ll doubt i will publish it

So this string I showed will be empty if the OS is not windows 10, you still have to add it to a legacy path , but it is not clear if the location you pin-pointed is the proper way to do it as maybe there would be other calls to file.getFullPathName() that would not get this prefix.

It would be more appropriate to apply this patch to juce::File instead, and once and for all.

Also it should preferably cache the long filename prefix in an opaque static string to call the statistics class only once.
in juce::File there is an internal “fullPath” member variable, this one should be once and for all prefixed with this longFilenamePrefix string I showed upper and it would not break other OSes.

EDIT: even better if you would add this prefix in

String File::parseAbsolutePath (const String& p)

It would be used everywhere now including the first fullPath internal string assignment.

Maybe try this new version for parseAbsolutePath:

String File::parseAbsolutePath (const String& p)
{
    if (p.isEmpty())
        return {};

   #if JUCE_WINDOWS
    static juce::String longFilenamePrefix(juce::SystemStats::getOperatingSystemType() >= juce::SystemStats::Windows10 ? "\\\\?\\" : "");

    // Windows..
    auto path = normaliseSeparators (removeEllipsis (p.replaceCharacter ('/', '\\')));

    if (path.startsWithChar (getSeparatorChar()))
    {
        if (path[1] != getSeparatorChar())
        {
            /*  When you supply a raw string to the File object constructor, it must be an absolute path.
                If you're trying to parse a string that may be either a relative path or an absolute path,
                you MUST provide a context against which the partial path can be evaluated - you can do
                this by simply using File::getChildFile() instead of the File constructor. E.g. saying
                "File::getCurrentWorkingDirectory().getChildFile (myUnknownPath)" would return an absolute
                path if that's what was supplied, or would evaluate a partial path relative to the CWD.
            */
            jassertfalse;

            path = File::getCurrentWorkingDirectory().getFullPathName().substring (0, 2) + path;
        }
    }
    else if (! path.containsChar (':'))
    {
        /*  When you supply a raw string to the File object constructor, it must be an absolute path.
            If you're trying to parse a string that may be either a relative path or an absolute path,
            you MUST provide a context against which the partial path can be evaluated - you can do
            this by simply using File::getChildFile() instead of the File constructor. E.g. saying
            "File::getCurrentWorkingDirectory().getChildFile (myUnknownPath)" would return an absolute
            path if that's what was supplied, or would evaluate a partial path relative to the CWD.
        */
        jassertfalse;

        return File::getCurrentWorkingDirectory().getChildFile (path).getFullPathName();
    }
   #else
    // Mac or Linux..

    // Yes, I know it's legal for a unix pathname to contain a backslash, but this assertion is here
    // to catch anyone who's trying to run code that was written on Windows with hard-coded path names.
    // If that's why you've ended up here, use File::getChildFile() to build your paths instead.
    jassert ((! p.containsChar ('\\')) || (p.indexOfChar ('/') >= 0 && p.indexOfChar ('/') < p.indexOfChar ('\\')));

    auto path = normaliseSeparators (removeEllipsis (p));

    if (path.startsWithChar ('~'))
    {
        if (path[1] == getSeparatorChar() || path[1] == 0)
        {
            // expand a name of the form "~/abc"
            path = File::getSpecialLocation (File::userHomeDirectory).getFullPathName()
                    + path.substring (1);
        }
        else
        {
            // expand a name of type "~dave/abc"
            auto userName = path.substring (1).upToFirstOccurrenceOf ("/", false, false);

            if (auto* pw = getpwnam (userName.toUTF8()))
                path = addTrailingSeparator (pw->pw_dir) + path.fromFirstOccurrenceOf ("/", false, false);
        }
    }
    else if (! path.startsWithChar (getSeparatorChar()))
    {
       #if JUCE_DEBUG || JUCE_LOG_ASSERTIONS
        if (! (path.startsWith ("./") || path.startsWith ("../")))
        {
            /*  When you supply a raw string to the File object constructor, it must be an absolute path.
                If you're trying to parse a string that may be either a relative path or an absolute path,
                you MUST provide a context against which the partial path can be evaluated - you can do
                this by simply using File::getChildFile() instead of the File constructor. E.g. saying
                "File::getCurrentWorkingDirectory().getChildFile (myUnknownPath)" would return an absolute
                path if that's what was supplied, or would evaluate a partial path relative to the CWD.
            */
            jassertfalse;

           #if JUCE_LOG_ASSERTIONS
            Logger::writeToLog ("Illegal absolute path: " + path);
           #endif
        }
       #endif

        return File::getCurrentWorkingDirectory().getChildFile (path).getFullPathName();
    }
   #endif

    while (path.endsWithChar (getSeparatorChar()) && path != getSeparatorString()) // careful not to turn a single "/" into an empty string.
        path = path.dropLastCharacters (1);

   // Add long prefix only once in Windows 10:
   #if JUCE_WINDOWS
    if (longFilenamePrefix.isNotEmpty() &&  !path.startsWith(longFilenamePrefix)) 
        path = longFilenamePrefix + path; 
   #endif   
    return path;
}

EDIT2: note the if condition in the end to keep compatibility with other OS’es and concat only once.

EDIT3: Finally, you should test more as I could test it does work for the Projucer app where I could visualize source code but there is a lot of other code you might want to check like revealInFinder() API and similar.

Ran:

       UnitTestRunner runner;
       runner.runAllTests();

Got successful report here, does not seem to break any test at least!
In particular:

Starting test: Files / Reading...
All tests completed successfully

The issue is that i’m on 10 and above that version and enabled long paths. but still using an empty prefix is failing. So i just added the prefix when needed in juce_win32_files.cpp and forced using W versions of windows API. Works great!

Glad to know it worked for you!

Yeah. Hope the juce team will sort it out for everybody

Replacing all the Win32 file calls with the unicode “W” versions shouldn’t be necessary since we define UNICODE in juce_BasicNativeHeaders.h, so Windows will resolve all the non-unicode calls to their unicode equivalents. Checking for the Windows version at runtime shouldn’t be needed either - as far as I can tell from the docs, the only difference on 1607+ is that the MAX_PATH limitation can be removed without prepending “\\?\” to the path but involves the user setting a registry key and including the longPathAware element in the app manifest.

To enable path names > MAX_PATH it looks like we need to prepend “\\?\” in all the Win32 file operations, but finding all of these and making sure that the behaviour doesn’t change is going to be a fair bit of work (see the “Win32 File Namespaces” section of https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file). The team is busy with ADC setup and other library features currently, but we’ll add this to the backlog.

1 Like

Sounds great so even easier that I thought, I updated my code example upper to remove the wide char spec.