Here is an interesting issue that came from a reader. He is using Yahoo's stock quote system to get data using CFHTTP. Here is the code he used:
<cfhttp name="Stocks" method="get" delimiter=","
firstRowAsHeaders="false" url="http://quote.yahoo.com/d/quotes.csv"
columns="stockID,IndexValue,TradeTimeDate,TradeTimeTime,ChangePoint,OpenValue,En
dingDayRange,OpeningDayRange,DoNotKnow1,MarketCap,PreviousClose,ChangePercent,Ye
arRange,EarningPerShare,EarningRatio,CompanyName"
throwonerror="yes">
<cfhttpparam type="Header" name="charset" value="ISO-8859-1" />
<cfhttpparam name="s" value="#form.symbol#" type="formfield" encoded="no"
/>
<cfhttpparam name="f" value="xxx"
type="formfield" />
</cfhttp>
I've not used the Yahoo API before, but it should be pretty obvious what the above does. Pass in a symbol via a form variable and parse the result into a query. Note - the "f" param was a longer string and I was concerned that it may be a private key. I changed it to xxx just to be safe.
So what's the problem? When he used ^DJI as a symbol, the value ended up being escaped. Yahoo's result indicated that it didn't recognize the value, which makes sense as it was no longer ^DJI, but had changed to %5EDJI. As you can see, he did pass encoded="no", but this was ignored. I thought perhaps it was the use of GET as the cfhttp operation. Changing this to POST did not help. Nor did changing the cfhttpparam type to URL. Nothing seemed to work right.
I tried a few different combinations, and finally got it working, but to be honest, I have no idea why this works. Consider the new version (and again, 'f' has been changed in case it is a private variable):
<cfhttp method="post"
url="http://quote.yahoo.com/d/quotes.csv?s=#urlEncodedFormat(form.symbol)#&f=xxx"
throwonerror="false"
columns="stockID,IndexValue,TradeTimeDate,TradeTimeTime,ChangePoint,OpenValue,EndingDayRange,OpeningDayRange,DoNotKnow1,MarketCap,PreviousClose,ChangePercent,YearRange,EarningPerShare,EarningRatio,CompanyName"
delimiter=","
name="Stocks"
firstRowAsHeaders="false"
>
<cfhttpparam type="Header" name="charset" value="ISO-8859-1" />
<cfhttpparam name="s" value="#form.symbol#" type="formfield" />
<cfhttpparam name="f" value="xxx" type="formfield" />
</cfhttp>
Notice I've supplied the s and f values both in the URL and in the form field. This worked perfectly, but as I said, I've got not idea why. Anyone want to take a guess?
Archived Comments
so if you remove the f and s form params and keep just in the URL sting it doesn't work?
I would have guessed that the URL string forces it to decode and thus work properly vs form which does not. but it would be odd that it would require both url and form to force that behavior.
I may not have made it clear. I needed both args, in both places, for it work. It didn't work w/o them both there.
That's bananas.
My bet is that they do a check for form fields in their code or reject it, but actually let the URL string work.
Try putting junk in the form field part.
The "f" param tells Yahoo! what information you want back from the query. More information here: http://www.gummy-stuff.org/...
Also to be clear this is not part of Yahoo!'s API, instead it is a free service as long as you state that the stock quotes are via Yahoo!
@JC: Nope, no go.
@Mike:Ah, sorry then. For folks who want to try my code, change F to: sl1d1t1c1ohgvj1pp2wern
This is the same "service" that the cf_stockgrabber tag I wrote back in 1997 uses. Interestingly enough, the MX version of the tag still works today.
My code is a little different, but the CFHTTP call looks like this:
<code>
<cfhttp
method="get"
url="http://download.finance.yah..."
name="myQuery"
columns="Symbol,Company_Name,Last_Traded_Price,Last_Traded_Date,Last_Traded_Time,Change,Opening_Price,Days_High,Days_Low,Volume"
delimiter=","
textqualifier=""""
firstRowAsHeaders="no">
</code>
Going this way, there's no problem with escaped characters.
Interesting. Rob, did you notice your URL was different? When you switch the URL to Mikes, and use the rest of your code, it breaks again.
Interesting indeed. I'm not sure where Mike's url came from. I've been using the one I posted forever. Tell him to switch ;-)
Try changing the "type" attribute in teh CFHTTPPARAM tags to "url". This is how my code to retrieve Yahoo! Stock Quotes is written (also, it is sent as a "GET"), and I've never had a problem with any symbols being sent.
Eric, I'm pretty sure I tried that and it failed. Does it work for you using the code above (fixing 'f' of course).
It's because your using the POST method in the 2nd example. In the first sample, you're using GET--which will escape the variables automatically (since GET is sent with the URL--not in the body of the request.)
Dan, trust, I tried switching to POST in the first example. I think I even say that in the blog post, don't I?
Perhaps the private key is required via the URL scope and the stock is expected in the FORM scope.
It might be the URL is just supposed to be:
http://quote.yahoo.com/d/qu...
Nope, see above comments. I was wrong about f. F helps define the results.
That "F" string is actually telling it which fields you want... there's no special codes, you can get the URL just by going to the yahoo finance site and saying you want to download the info.
same service, this code works and has for ages:
<cfparam name="attributes.symbols" default="^DJI">
<cfhttp URL="http://finance.yahoo.com/d/..."
METHOD="GET"
PORT="80"
RESOLVEURL="true"
timeout="120">
I'll cut most of the rest out to keep it short.
<cfset detailarray = listtoarray(cfhttp.filecontent)>
<cfscript>
symbol = detailarray[1];
date = detailarray[2];
tradetime = detailarray[3];
lasttrade = detailarray[4];
askprice = detailarray[5];
bidprice = detailarray[6];
start52 = detailarray[7];
end52 = detailarray[8];
daystart = detailarray[9];
dayend = detailarray[10];
volume = detailarray[11];
previousclose = detailarray[12];
change = detailarray[13];
lastopen = detailarray[14];
</cfscript>
err
URL="http ://finance.yahoo.com/d/quotes.... s=#attributes.symbols# &f=sd1t1l1abjkghvpc1o &e=.csv"
strip the spaces out
sorry to post too much... but here:
http://finance.yahoo.com/q?s=^DJI
righthand side, beneath the chart, it says "Download Data" and that's where this link comes from:
http://download.finance.yah...
it's not a post, it's a get. Just a plain old link on a webpage.