URL Https redirects


#1

I am having a problem with http -> https redirects being ignored on Linux with CURL. I am using the github API as an example as it forces use of https. This means if you access the http version with curl from CMD line you get a nice redirect:

$ curl -L -i http://api.github.com/users/octocat/orgs                             
HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https://api.github.com/users/octocat/orgs
Connection: close

HTTP/1.1 200 OK
Server: GitHub.com
Date: Wed, 06 Jul 2016 18:14:57 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 5
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 54
X-RateLimit-Reset: 1467830696
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "98f0c1b396a4e5d54f4d5fe561d54b44"
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
X-Served-By: dc1ce2bfb41810a06c705e83b388572d
X-GitHub-Request-Id: C1CB577A:7ABD:A91D51D:577D4AA0`

I have JUCE code that looks something like this:

std::cout << "GitHub over https: " 
          << URL("https://api.github.com/users/octocat/orgs").readEntireTextStream()
          << std::endl;

With JUCE_USE_CURL turned on on linux I get an error message that I don’t care about (I haven’t set my User-Agent headers properly but that isn’t the point):

GitHub over https: 
Request forbidden by administrative rules. Please make sure your request has a User-Agent header (http://developer.github.com/v3/#user-agent-required). Check https://developer.github.com for other possible causes.

With the https link replaced with http (which should redirect only once to https) I don’t get any returned data at all. I tried to debug it a bit but I didn’t have a lot of time.


#2

Thank you for reporting. This should be fixed on the develop branch.


#3

Hi @fabian,

My bug isn’t in the User Agent (though this is what Github is moaning about - sorry for the confusion). My bug is that if I try to access the http version of the github api (which with curl -L from command line happily redirects to https version) it doesn’t receive any data.

i.e. with http access from curl cmd line

$ curl -i -L http://api.github.com/users/octocat/orgs
HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https://api.github.com/users/octocat/orgs
Connection: close

HTTP/1.1 200 OK
Server: GitHub.com
Date: Thu, 07 Jul 2016 10:43:02 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 5
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 54
X-RateLimit-Reset: 1467891579
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "98f0c1b396a4e5d54f4d5fe561d54b44"
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
X-Served-By: e724c57ebb9961c772a91e2dd7421c8d
X-GitHub-Request-Id: C1CB577A:7ABD:B48E574:577E3236

[

]

and with:

std::cout << "GitHub over http with redir: " 
          << URL("http://api.github.com/users/octocat/orgs").readEntireTextStream()
          << std::endl;

you get:

$ ./build/Test 
JUCE v4.2.3
GitHub over http with redir:

#4

Ahh. I see. I think this should be fixed now on develop.


#5

Thanks @fabian, seems to be solved.