Twitter: raymondcamden


Address: Lafayette, LA, USA

Important note about ColdFusion 11 and CFHTTP

05-22-2014 3,922 views ColdFusion 12 Comments

As a quick aside before I begin, multiple people were involved in this research. I thank them all at the end, but I wanted to be sure folks know this was definitely something I got a lot of help on.

Earlier this week a user reported an odd issue to me. He was using CFHTTP to hit the Google Places API and noticed that code he had in ColdFusion 10 worked, but in ColdFusion 11 the same code returned a 404 error. I was able to quickly confirm the issue and began my investigation.

The first thing I noticed was that the URL in question also worked fine in the browser. I seemed to remember an API a few years ago that blocked the default user agent used with CFHTTP so the first thing I attempted was a simple change to that value. I got my own user agent and supplied it to the tag, but it didn't help.

Next - I was curious if there was some particular change to the headers being sent by CFHTTP between versions. To test this, I wrote a new CFM that dumped the headers: <cfdump var="#getHTTPRequestData()#">

When I did so, I noticed two changes. First, ColdFusion 11 was sending Accept-Encoding with gzip, deflate. Secondly, the host value recorded was localhost:80 compared to localhost on ColdFusion 10. I assumed the second change wasn't relevant. My CF11 box was localhost:8511 and since I was hitting localhost/test.cfm, I figured it was just noting the different port. As to the first change, I added that header to the ColdFusion 10 server test and after confirming the headers matched, switched back to hitting Google and was not able to replicate the bug.

So - here is where things get interesting. Turns out, the change with the port number was important. Rupesh from the ColdFusion team confirmed that the HttpClient library used in ColdFusion 11 always adds the port to the URL, even when you don't put it there. While a host of foo should be the exact same as foo:80, Google, for whatever reasons, doesn't like the port and returns a 404. (Dan Switzer made the argument, though, that for OAuth 1 calls every part of the URL is significant, and the port would make a difference there.)

So - Rupesh has filed a bug report with Apache in regards to HttpClient, which is good, and he has stated that they will try to make a workaround for this in ColdFusion itself. To be clear, you do not need to worry about this for - I'd guess - a majority of your cfhttp calls. But you definitely want to be aware of it.

Luckily, there is a fix. A super easy fix. I only found this after I wrote a fix in Java, and kicked myself for missing this email from David Boyer. Just add a Host header and everything works: <cfhttpparam name="Host" value="maps.googleapis.com" type="header">

Thanks to David Boyer, Dan Switzer, Mark Kruger, Rupesh, Wil Genovese, and anyone else I forgot.

12 Comments

  • Ben Reid #
    Commented on 06-03-2014 at 1:25 AM
    We have also experienced this issue and have not yet managed to find a work-around.

    We are connecting to non-standard ports (not port 80 or 443) on localhost.

    I tried your suggestion of providing the http header for HOST, but that did not work for us.

    The URL we use is something like http://localhost:9292/path/

    Note also that we are not requesting a specific file in the URL.

    Please re-post if you find out more information on this.
  • Commented on 06-03-2014 at 9:35 AM
    I'm not able to replicate that. I've got a Node server up on 3333 and my CF11 server can cfhttp it just fine. Sorry.
  • Ben Reid #
    Commented on 06-04-2014 at 11:02 PM
    I have since worked out that the 3rd-party web server we are trying to connect to is a Jetty v6.1.14 http server (which was released in November 2008).
  • Ben Reid #
    Commented on 06-04-2014 at 11:05 PM
    Also, whilst the GET request is failing with "I/O Exception: null", making a HEAD request responds with "200 OK", and making an POST request responds with "405 Method Not Allowed".
  • Commented on 06-11-2014 at 11:15 AM
    Ray - FYI the bug on for HttpClient is here: https://issues.apache.org/jira/browse/HTTPCLIENT-1... and they marked it as Resolved - Not a Problem. So it looks like they are leaving it up to adobe to fix on their end.
  • Commented on 06-11-2014 at 11:21 AM
    I wonder what their logic is. (They being the Apache-side folks, not the CF team.)
  • Commented on 06-11-2014 at 11:31 AM
    It sounds to me like CF is always appending the port to the url and then passing that url to HttpClient. The guy that commented on the bug seams to be saying that if you don't put the port in the url it won't add it to the host header.
  • Commented on 06-11-2014 at 11:32 AM
    Hmmm. I thought CF wasn't, but the library was?
  • Private Uly #
    Commented on 06-26-2014 at 11:13 PM
    Ray,

    Thanks for this blog post. It saved me much hair pulling. We had a problem connecting with amazon mws from CF11, except that the error message that came back was that the request signature was wrong. So I spent quite a bit of time debugging that part to no avail. After I added the host parameter to the cfhttp everything was fine again.

    It's not clear who is appending the port number. Are CF10 and CF11 running the same version of HttpClient? If it's CF11 then Adobe should fix it.
  • Commented on 06-27-2014 at 9:02 AM
    From what I know it was Apache doing it, but they closed the bug and said it was our fault. Not sure I agree with that, but from what I know, we will be doing a workaround. (Not that I speak for the CF team.)
  • Commented on 07-29-2014 at 9:36 AM
    Thanks for the blog post.

    We have an application that makes numerous API calls and have found several that fail with the 404 error.

    In some cases we have resolved the issue by removing URL variables from the web address and placing them as CFHTTPPARAM tags instead.

    For instance,
    <cfhttp url="http://www.apicall.com/?key=xxx"; method="get" result="apiResult"></cfhttp>

    Would be changed to:
    <cfhttp url="http://www.apicall.com/"; method="get" result="apiResult">
    <cfhttpparam type="url" name="key" value="xxx">
    </cfhttp>

    One such API where this worked is: url2png.com.

    Hope this helps someone.
  • Jon Kocen #
    Commented on 08-12-2014 at 2:18 PM
    I am trying to CFHTTP to a SharePoint site and get the dreaded 404 Not Found. I can't use the CFSHAREPOINT because we are all SSL. I have tried all of the above suggestions with no luck. I can CFHTTP to other servers with success.

    Another other tips/tricks?

Post Reply

Please refrain from posting large blocks of code as a comment. Use Pastebin or Gists instead. Text wrapped in asterisks (*) will be bold and text wrapped in underscores (_) will be italicized.

Leave this field empty