URL File Upload of large files vs app memory

MacOSX and iOS
1

Hi there

When trying to use:
url.readEntireTextStream( “true” );
to upload a file, the app seems to need to read the whole file into memory. When dealing with large files this quickly causes the debugger to kill the app for using too much memory.

Is there a way to upload the file, but avoid reading the entire file into memory first?

Many thanks and best wishes

Jeff

3

In researching this I was also looking at c++ ways for splitting up a file into chunks ahead of time. Transmitting those and joining them on the server side.

Seems far less than ideal but it’s an option I suppose.

Also I was looking at how to combine swift and juce/c++ in a single app. Definitely outside of my abilities currently and ahead, a pretty brutal solution.

Any thoughts please anyone?

4

How about using url.downloadToFile?

5

Hey @cpr, for uploading to a php server? I didn’t think to try that. From the name “download” that doesn’t seem like it would work though. url.uploadFromFile would be much more hopeful sounding :wink: lol

6

Uploading? readEntireTextStream is also downloading… it reads from a server… maybe I am somehow misunderstanding what you are doing… From the docs:
Tries to download the entire contents of this URL as a string.

7

Yeah. Took a while to figure this out but apparently url.readEntireTextStream(“true”) is a valid way to upload files.

void fileTransfer(File file)
{
    URL urlUp = URL("http://192.168.0.255/uploaderScript.php")
    
    url = urlUp.withFileToUpload(file.getFileName(), File(file.getFullPathName()), "video/quicktime");

    
    result = urlUp.readEntireTextStream(true); //actually uploads here
}

The problem is the whole file needs to be read into memory which can easily trigger a crash if it’s a big file.

8

@cpr I looked at downloadToFile but didn’t have any joy with it. Have you managed to use downloadToFile to upload a file to remote location? Seems unintuitive to me if so!

9

As I said, I misunderstood what you were doing when I suggested 'downloadToFile. I did a bunch of URL code a couple of years ago, and wrote a pretty robust system which hid a bunch of the details, and so I don’t remember all of the details. I do know that I ended up using WebInputStreams on my own thread to manage transfers. I was doing simple REST API type stuff, along with uploading/downloading large audio files without any issues. My code also loads the entire file for uploading.

10

@cpr Hey dude, sorry I should have been way more clear. This is all on an iOS device hence I’m running out of memory so easily. It seems like trying to stream the upload rather than load all into memory first is the way to go. I’m trying to see how to go about doing that :slight_smile: