URL::createInputStream(...)


#1

I’m creating a connection from a thread on an audio plugin.

If the connection is still delayed and I need to stop the thread it’s connecting from executing, how do I safely stop this?

The callback (on windows) doesn’t seem to be being called while the connection is outstanding. It’s blocking in HttpSendRequestEx.

Even with the timeout set to 2 seconds, and the stopThread timeout set to 6 seconds I’m still hitting the bad-karma kill thread assertion. If I increase the timeout to 16 things exit nicely but unacceptably slowly, and the http timeout is very tight.


#2

Don’t you just put StopThread() in the destructor of your processor?


#3

Could you use the progressCallback argument in createInputStream and return false from the callback if you need to abort?


#4

It never gets called, because the OS is blocking in HttpSendRequestEx


#5

I can stop the thread for sure … but I think that’s not exactly graceful and may leave other problems behind…


#7

Other problems (from MSDN):

TerminateThread is used to cause a thread to exit. When this occurs, the target thread has no chance to execute any user-mode code. DLLs attached to the thread are not notified that the thread is terminating. The system frees the thread’s initial stack.
Windows Server 2003 and Windows XP: The target thread’s initial stack is not freed, causing a resource leak.
TerminateThread is a dangerous function that should only be used in the most extreme cases. You should call TerminateThread only if you know exactly what the target thread is doing, and you control all of the code that the target thread could possibly be running at the time of the termination. For example, TerminateThread can result in the following problems:
If the target thread owns a critical section, the critical section will not be released.
If the target thread is allocating memory from the heap, the heap lock will not be released.
If the target thread is executing certain kernel32 calls when it is terminated, the kernel32 state for the thread’s process could be inconsistent.
If the target thread is manipulating the global state of a shared DLL, the state of the DLL could be destroyed, affecting other users of the DLL.


#8

I’ve now hit the issue you’re speaking of. With the URL inputstream blocking the check for threadShouldExit, it’s quite difficult to have an elegant and timely exit. I’ll report back here if I find a nice work around.


#9

So, here is how I think you’re meant to handle it. Forgive me if I’m wrong, but for me, the progress callback is getting called.

you declare a static function in your header to pass as the argument. So in your header something like:

static bool progressCallback (void *context, int bytesSent, int totalBytes);

then in your implementation:

bool YourClass::progressCallback(void *context, int bytesSent, int totalBytes)
{
    if (static_cast<YourClass*>(context)->shouldExit())
        return false;
    else
        return true;
}

and when you open your stream pass in your callback and thread in as the arguments:

 URL::OpenStreamProgressCallback* callback = &YourClass::progressCallback;
        
        InputStream = URL.createInputStream(false, callback, this, String::empty, 0, &headerResponse, &statusCode);

I’m just diving into this, but hopefully this should be the solution.


#10

yikes… actually… progress callback is only called when the connection is being opened and sending data, however if a download is happening it stops calling the callback… perhaps we need callbacks to check exit calls for downloads as well…


#11

It’s not called when it’s being opened either, just during transmission.

Having dug into the Windows API a bit it looks like that might be the best that can be done using those particular functions.

I think I’m going to integrate libcurl into my project to get a bit more control over all this.


#12

Yes the progress callback is just for sending large POST data. If you want to track the progress in the other direction then you just monitor how many bytes you have already read from the input stream vs. the total length of the input stream (i.e. inputStream->getTotalLength()). You can do this from another thread.


#13

You don’t know how to get it to abort safely if it’s still trying to connect though do you, e.g. if the destination server is not responding to a SYN request…?


#14

Well yes you can’t really. We may need to add more callbacks or modify the progress callback. I’ll look into this.


#15

Sadly, I think the Windows API may prevent a good solution here. At least not without a completely different approach to the whole thing…


#16

Can you try using libcurl instead. JUCE supports libcurl. Simply enable CURL support in the Projcuer’s juce_core module settings.


#17

Oh shit - I’d forgotten that was that easy…I’ll be doing that instead then and let you know if I run into any problems…


#18

Well, the issue for me is that while the inputstream reads into a memory block, the ThreadPoolJob hangs on that line and stops checking whether or not it should exit.

What becomes available if one was to switch to using LibCurl internally? Is there extra functionality exposed?


#19

I think some API changes will really be necessary. One option is to return an InputStream as early as possible, even before JUCE actually tries to contact the server. Deleting this InputStream would cancel the request. But this would be a breaking change as many JUCE developers probably rely on a non-null return value of createInputStream indicating success.

I guess we need to implement some more fine-grained APIs.


#20

Well, actually for my problem, this works well:

 while (!InputStream->isExhausted() && !shouldExit())
 {
     InputStream->readIntoMemoryBlock(memory, 1000);
 }

I thought the thread was hanging on opening and downloading the stream, but it was actually while reading into the block, now the threads are exiting fine & quickly. Sorry if this is not the same issue as you @bazrush, I’m not sure what issues could arise to simply setting the stream to a nullptr, but the threads are exiting cleanly and quickly.


#21

if createInputStream can hang while it tries to contact the server it isn’t really suitable for being called on the UI thread at all. And that means you’ve got to run it on some other thread. And then there doesn’t seem to be any way to abort the initial handshake without killing the thread (or waiting for a potentially long timeout).

I think some API changes will really be necessary. One option is to return an InputStream as early as possible, even before JUCE actually tries to contact the server. Deleting this InputStream would cancel the request. But this would be a breaking change as many JUCE developers probably rely on a non-null return value of createInputStream indicating success.

Well, that’s the other problem. Error handling. It’s lacking in detail in the URL class. As far as I know there’s no way of reporting the difference between a connection refused, timed-out, DNS lookup failure etc. Presumably we could do some really nice mapping of OS specific errors to a cross-platform equivalent.

(createInputStream could fail anyway though, immediately after connecting?)

It’s really good class for getting something quick and dirty working with though :slight_smile: