As a quick aside before I begin, multiple people were involved in this research. I thank them all at the end, but I wanted to be sure folks know this was definitely something I got a lot of help on.
Earlier this week a user reported an odd issue to me. He was using CFHTTP to hit the Google Places API and noticed that code he had in ColdFusion 10 worked, but in ColdFusion 11 the same code returned a 404 error. I was able to quickly confirm the issue and began my investigation.
The first thing I noticed was that the URL in question also worked fine in the browser. I seemed to remember an API a few years ago that blocked the default user agent used with CFHTTP so the first thing I attempted was a simple change to that value. I got my own user agent and supplied it to the tag, but it didn't help.
Next - I was curious if there was some particular change to the headers being sent by CFHTTP between versions. To test this, I wrote a new CFM that dumped the headers: <cfdump var="#getHTTPRequestData()#">
When I did so, I noticed two changes. First, ColdFusion 11 was sending Accept-Encoding with gzip, deflate. Secondly, the host value recorded was localhost:80 compared to localhost on ColdFusion 10. I assumed the second change wasn't relevant. My CF11 box was localhost:8511 and since I was hitting localhost/test.cfm, I figured it was just noting the different port. As to the first change, I added that header to the ColdFusion 10 server test and after confirming the headers matched, switched back to hitting Google and was not able to replicate the bug.
So - here is where things get interesting. Turns out, the change with the port number was important. Rupesh from the ColdFusion team confirmed that the HttpClient library used in ColdFusion 11 always adds the port to the URL, even when you don't put it there. While a host of foo should be the exact same as foo:80, Google, for whatever reasons, doesn't like the port and returns a 404. (Dan Switzer made the argument, though, that for OAuth 1 calls every part of the URL is significant, and the port would make a difference there.)
So - Rupesh has filed a bug report with Apache in regards to HttpClient, which is good, and he has stated that they will try to make a workaround for this in ColdFusion itself. To be clear, you do not need to worry about this for - I'd guess - a majority of your cfhttp calls. But you definitely want to be aware of it.
Luckily, there is a fix. A super easy fix. I only found this after I wrote a fix in Java, and kicked myself for missing this email from David Boyer. Just add a Host header and everything works: <cfhttpparam name="Host" value="maps.googleapis.com" type="header">
Thanks to David Boyer, Dan Switzer, Mark Kruger, Rupesh, Wil Genovese, and anyone else I forgot.
Archived Comments
We have also experienced this issue and have not yet managed to find a work-around.
We are connecting to non-standard ports (not port 80 or 443) on localhost.
I tried your suggestion of providing the http header for HOST, but that did not work for us.
The URL we use is something like http://localhost:9292/path/
Note also that we are not requesting a specific file in the URL.
Please re-post if you find out more information on this.
I'm not able to replicate that. I've got a Node server up on 3333 and my CF11 server can cfhttp it just fine. Sorry.
I have since worked out that the 3rd-party web server we are trying to connect to is a Jetty v6.1.14 http server (which was released in November 2008).
Also, whilst the GET request is failing with "I/O Exception: null", making a HEAD request responds with "200 OK", and making an POST request responds with "405 Method Not Allowed".
Ray - FYI the bug on for HttpClient is here: https://issues.apache.org/j... and they marked it as Resolved - Not a Problem. So it looks like they are leaving it up to adobe to fix on their end.
I wonder what their logic is. (They being the Apache-side folks, not the CF team.)
It sounds to me like CF is always appending the port to the url and then passing that url to HttpClient. The guy that commented on the bug seams to be saying that if you don't put the port in the url it won't add it to the host header.
Hmmm. I thought CF wasn't, but the library was?
Ray,
Thanks for this blog post. It saved me much hair pulling. We had a problem connecting with amazon mws from CF11, except that the error message that came back was that the request signature was wrong. So I spent quite a bit of time debugging that part to no avail. After I added the host parameter to the cfhttp everything was fine again.
It's not clear who is appending the port number. Are CF10 and CF11 running the same version of HttpClient? If it's CF11 then Adobe should fix it.
From what I know it was Apache doing it, but they closed the bug and said it was our fault. Not sure I agree with that, but from what I know, we will be doing a workaround. (Not that I speak for the CF team.)
Thanks for the blog post.
We have an application that makes numerous API calls and have found several that fail with the 404 error.
In some cases we have resolved the issue by removing URL variables from the web address and placing them as CFHTTPPARAM tags instead.
For instance,
<cfhttp url="http://www.apicall.com/?key..." method="get" result="apiResult"></cfhttp>
Would be changed to:
<cfhttp url="http://www.apicall.com/" method="get" result="apiResult">
<cfhttpparam type="url" name="key" value="xxx">
</cfhttp>
One such API where this worked is: url2png.com.
Hope this helps someone.
I am trying to CFHTTP to a SharePoint site and get the dreaded 404 Not Found. I can't use the CFSHAREPOINT because we are all SSL. I have tried all of the above suggestions with no luck. I can CFHTTP to other servers with success.
Another other tips/tricks?
Thanks! This seems to resolve an issue I was having connecting to the Instagram API. Adding api.instagram.com as the value fixed the issue.
Hi Raymond - I was looking at this blog about CF11 CFHTTP issue and looks like I have the similar problem. I tried few solutions mentioned above with no luck. We are upgrading from CF9 to CF11 on LINUX with Apache web server. Our current application pages works fine in CF9 but if I use the same page in CF11 server, I'm getting "404 Not Found" with "COM.Allaire.ColdFusion.HTTPNotFound" error message. Thought it was CFHTTP issue but now I'm not so sure. Any pointers will be appreciated. - Kumar
I'd suggest calling up Adobe support.
I wish I had found this 4 hours ago! Thanks again, Mr. Camden.
I am trying to send sms messages via messagebird.com. On my old coldfusion 10 server everything was working fine but now we get Connection failure messages on coldfusion 11 update 7. I have tried
all of the above suggestions with no luck. Any suggestions? Anybody using messagebird with coldfusion 11?
How to use message bird API with ColdFusion?
This isn't on topic for this blog post.
Nice one Ray, I just noticed I'm having this issue with Github. A call to my user accounts repo JSON feed is failing with a Java error:
`I/O Exception: Received fatal alert: protocol_version`
Oddly enough though my host is running CF 10.x
Hope this helped then. :)