Earlier today I blogged about ColdFusion's Asynchronous Gateway. I promised a more appropriate example to demonstrate the power of the gateway, and instead of watching the Steelers lose, I figured I'd write code instead.
One note: I don't mean to imply that using the Asynch Gateway for logging is bad, or not a "real" example. Michael Dinowitz wrote a good comment about he uses asynch logging on his own site. Be sure to read his remarks.
So - the new demo. A while ago I wrote a proof of concept for Adobe's new cfthread/cfjoin tags (which may or may not be added to the product). The code simply fetched my RSS feed, grabbed each URL, and then downloaded them via CFHTTP to the local file system. Here is an example of the code using nothing special at all, just simple CFML:
<cfhttp url="http://ray.camdenfamily.com/rss.cfm" result="result">
<cfset myrss = result.filecontent>
<cfset myrssParsed = xmlParse(myrss)>
<cfset myurls = xmlSearch(myrssParsed, "/rss/channel/item/link/text()")>
<cfset links = arrayNew(1)>
<cfloop index="x" from="1" to="#arrayLen(myurls)#">
<cfset arrayAppend(links, myurls[x].xmlvalue)>
</cfloop>
<cfloop index="loopcounter" from="1" to="#arrayLen(links)#">
<cfhttp url="#links[loopcounter]#" result="result">
<cfset filename = getDirectoryFromPath(getCurrentTemplatePath()) & "agtest/" & loopcounter & ".html">
<cffile action="write" file="#filename#" output="#result.filecontent#">
<cfoutput>Done with #links[loopcounter]#<br></cfoutput>
<cfflush>
</cfloop>
The first half of the code handles creating an array of links. (And will not be repeated in my next example, but it will still be used.) I then loop over each item in the array, fetch it, and store the result. On average this took about 8-9 seconds on my machine.
Now for the Asynch version. I created a new gateway instance named URLSucker. Obviously you want to use the Asynchronous Gateway type. I pointed it to a new CFC. Lets take a look at that code:
<cfcomponent output="false">
<cffunction name="onIncomingMessage" output="false" returnType="void">
<cfargument name="cfEvent" type="struct" required="yes">
<cfset var result = "">
<cfif structKeyExists(arguments.cfEvent.data, "url") and structKeyExists(arguments.cfEvent.data, "filename")>
<cfhttp url="#arguments.cfEvent.data.url#" result="result">
<cffile action="write" file="#arguments.cfEvent.data.filename#" output="#result.fileContent#">
</cfif>
</cffunction>
</cfcomponent>
This is rather short so it should be easy to explain. I mentioned in the last entry that your CFC has to have a method named onIncomingMessage. The data that I pass to the gateway will exist in the arguments.cfEvent.data value. For URLSucker, there are two values - a URL and a filename. I fetch the HTML and save it. Now let's go back to our CFM and see how the new calls work:
<cfset props = structNew()>
<cfloop index="loopcounter" from="1" to="#arrayLen(links)#">
<cfset filename = getDirectoryFromPath(getCurrentTemplatePath()) & "agtest/a_" & loopcounter & ".html">
<cfset props.url = links[loopcounter]>
<cfset props.filename = filename>
<cfset status = sendGatewayMessage("URLSucker", duplicate(props))>
<cfif status>
<cfoutput>Done with #links[loopcounter]#<br></cfoutput>
<cfflush>
</cfif>
</cfloop>
If you remember, the gateway expects a struct of data. I create it once before the loop since I will just be changing the values inside the loop. I create the URL and filename values I mentioned that my CFC will be using. I then use sendGatewayMessage, telling it to use URLSucker and passing in my structure. (Note the duplicate() call, I'll come back to it later.)
So - at this point, I've basically taken the slow part of my code out of the CFM and I'm letting the asych gateway handle running it. If you run the CFM again it will be incredibly faster. If you then monitor the agtest folder (that's where I stored the results), you will see the files pop up a few seconds after the CFM is run.
Pretty easy, eh? What's up with the duplicate() call then? Well, structs passed to functions are actually passed by referenced. I knew this - but I thought that there was enough of a disconnect between my CFM and the CFC that the duplicate wouldn't be needed. But without it, all your files end up using the filename and url of the last entry in the loop. Very surprising, but also very easy to fix.
p.s. As I wrote this, the Steelers have now taken the lead. Someone remind me to blog next time the Saints play.
Archived Comments
I'm sorry, can't help you. Unfortunetly I live in Chicago and by definition must be a Bears fan (lucky me).
I tell ya - this is their year. They are going all the way. Really. They won't play all their games and start off by doing ok and then just completely giving up in the second half. Really.
OK, so the Async Gateway has a "que" of sorts that can stack the calls within the loop? How does it keep them separate? I hate asking dumb questions, but hey, gotta learn somehow. It's hard to grasp the concept of this gateway when you think of traditional code processing.
By the way...STEELERS?? You're killing me dude. And the SAINTS? Do you just HATE Cleveland?
Oh well, I've always got Ohio State.
I don't think it is queud up per say but ran all at once. In other words, imagine me putting ten letters in the mail box. When picked up by USPS, they all move at once.
Except if the CF Admin setting "Event Gateway Processing Threads" is set to, say, 5 then USPS will collect 5 and then when it has delivered one it would go back and collect another 1 etc until there were no more left in the mailbox.
Thanks for the clarification dickbob!
Hi Ray
Any specific reason you didn't use the path or file attribute of <cfhttp> tag & used <cffile action="write">?
Also if you are not happy using duplicate you could also initialize your props inside the <cfloop>. Just another way of doing it.
Rahul: Yes, I forgot. ;) As for the loop - I thought it seemed silly to have to do a structnew for each case when I was just changing the values, but that could certainly be done instead.
The pass-by-reference trick is how my Concurrency library works (that allows you to call methods on CFCs asynchronously *and* get the results back when the methods are done). Download from the 'software' pod on my blog (well, you need SVN now since I moved it to Google Code). I'll make a build downloadable from the Google Code page one day :)
Model-Glue can use my Concurrency library to allow listeners to be asynchronous (it creates a FutureTask from my library and puts it in session scope so you can test when the listener has actually completed and take appropriate action several pages later in the app!).
I had meant to mention your Concurrency lib, so thanks for doing that. I'm going to mention it again my third post. (Or as I call it, "Asynch Gateways - Everything Else")