My little project -

This post is more than 2 years old.

I've been hinting at a little project the last few days and as I'm about to get on an airplane I thought now would be a good time to share. First let me share the URL and I know folks will just immediately link out, so please do me a favor and open this in a new window. I want to talk a bit about the project before you leave.

So first and foremost, let me just say that I'm not trying to compete with MXNA, Goog, or Feed Squirrel. For a few days now I've been thinking of how easy it would be to build a blog aggregagtor in ColdFusion 8. I decided to build a site that would serve as a proof of concept. So to be clear, this site is mainly for fun and to showcase ColdFusion 8 technology. With that in mind, I've provided a link in the FAQ to the source. You can download all the code and play around with it. Outside of that, be gentle. Again, if you want a real aggregator, you know where to go. If enough folks like it though I'll consider adding more features. (The "Preferences" page for example is totally empty.)

So with that in mind, let me talk a bit about the code behind this site. First and foremost, in my mind, a blog aggregator does this:

  1. Download RSS entries from various feeds
  2. Add unique entries into the database
  3. Present a "blended" view to the user.

Pretty simple, right? I started off working with Paragator. Paragator is my CFC based Aggregator function. Basically it takes an array of RSS feeds and grabs them all at once, using threading. (Well, not all at once, the server has a max number of threads.) So having the CFC meant 90% of my work was done.

Then I wrote a simple CFC to manage my blogs and entries. I used the OPML (list of blogs) feed from MXNA to seed my list of blogs. I then wrote methods to get all the blogs. This was then passed to Paragator (just a small note, I'm running a slightly modified version of the code). When the results come back, I use a method that will only insert new entries.

And that's it. 2 CFCs and a process page. Oh and one page to dump the results. Incredibly simple. The only really cool tags I used were cffeed and cfthread.

On the front end I did a bit of AJAX work. The Feeds page makes use of CF8 tabs. The main page uses cftooltip on the blog names to show their descriptions. I want to add a bit more AJAX to the front end, but I just don't know exactly where yet.

So thats it. Again - I wrote this for fun. If it takes off, great (another chance to slap some ads up! ;), but I mainly wanted to show off ColdFusion 8. Download the code and let me know what you think. And to be clear - my readers know my UI skills stink. The design you see on the blog is an open source one made by styleshout. (I have to admit - I liked it so much I almost reskinned this blog with it.)

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA

Archived Comments

Comment 1 by Sam Farmer posted on 7/20/2007 at 1:12 AM

Very cool!

Great use of new CF8 features as well.

Maybe the MXNA folks could take your code and make MXNA run faster?

Comment 2 by Ben Nadel posted on 7/20/2007 at 1:16 AM

Ha ha ha:

"Currently 33 users are wasting time instead of working."

Ray, looks good. I assume it has some sweet CFThread and CFFeed usage. Can't wait to download and take a peek inside.

Comment 3 by Russ Johnson posted on 7/20/2007 at 1:19 AM

Very Nice Ray! My only complaint is that the titles of the posts dont stand out quite enough especially in the gray background posts.. maybe make the titles bold?

Keep up the great work!

Comment 4 by John Ramon posted on 7/20/2007 at 1:59 AM

Nice work Ray.

Comment 5 by Brian Rinaldi posted on 7/20/2007 at 2:59 AM

Nice work Ray. A question...what did you use to determine uniqueness? I think MXNA uses the GUID which changes whenever people update their blogs across versions or software which makes for the fairly regular "blog puking" as I call it when all the old posts suddenly show up at once. Goog uses the title from what I can tell which means that if you have two posts with the same title, the newer one doesn't show (which I noticed when my original open source update posts had the same name thus I added the date). Curious how you approach this problem.

Comment 6 by Dale Fraser posted on 7/20/2007 at 3:35 AM

Good job Ray,

I have been on MXNA for a while, and i'm on this one, Full As A Goog refuse to index me, I don't think they appreciate constructive critism.

I'm going to add it to my iGoogle feeds right away.

Comment 7 by Rey Bango posted on 7/20/2007 at 5:03 AM

Awesome work Ray! If you can, please add a pingurl feature like MXNA which will automatically alert the aggregator of a new entry.

Comment 8 by John Ramon posted on 7/20/2007 at 7:00 AM

Hey Ray I think there may be a little problem, if you search for something and it returns one item it's not displayed.

If you search for "My little project -" It returns one item but doesn't display anything.

Comment 9 by Raymond Camden posted on 7/20/2007 at 8:59 AM

John, are you saying search fails if it returns 1 result only but works ok for 2+?

Comment 10 by Mikkel Johansen posted on 7/20/2007 at 9:46 AM

WOW - It's very fast !!

I know that CF8 is fast, but it would be nice to get the specs on the server the site is running on.

Comment 11 by Raymond Camden posted on 7/20/2007 at 4:13 PM

It's a dual core something or another w/ 2 gigs of ram.

Comment 12 by John Ramon posted on 7/20/2007 at 7:08 PM

It's doesn't work correctly if you search "getter" 8 results are returned but if you count them you'll see the script only outputs 7 posts. So if one result is returned nothing is displayed but the title show one result found.

Comment 13 by Adam posted on 7/20/2007 at 7:25 PM

Ray - Yes, John is correct. Another example: search for "onMissingTemplate Example"

Comment 14 by Raymond Camden posted on 7/20/2007 at 9:41 PM

Confirmed. Working on a fix now. I think its just a issue with my limit in the SQL.

Comment 15 by Raymond Camden posted on 7/20/2007 at 10:02 PM

Should be fixed now. The source zip isn't updated.

Comment 16 by John Ramon posted on 7/20/2007 at 10:16 PM

Cool Ray, can yuo updated us when it is updated. I would like to see the fix.

Comment 17 by Raymond Camden posted on 7/20/2007 at 10:20 PM

I can tell you now I will forget. The first was that in MySQL limit clauses, the start index is 0, not 1. Duh. I will NEVER accept that starting crap at 0 makes sense. I think it was a mistake and modern languages should take a stand against this. (I mean other languages since CF did the sensible thing.)

Comment 18 by Charlie Arehart posted on 7/22/2007 at 6:58 PM

Cool stuff, of course, Ray. Here's an interesting point about such Ajax-driven interfaces, though.

Let's say one wanted to bookmark the results of a particular search. We can't, because the URL for the page never changed.

I suppose if one really did want to enable this, a small icon of some sort in the search pod could offer a link to a URL that would recreate the whole page and the given search string.

I don't expect you to change this, Ray. Just thought I'd share it while I notice.

Comment 19 by Raymond Camden posted on 7/22/2007 at 7:05 PM

I don't quite get your meaning here Charlie. I do re-use index.cfm for my search results, just like BlogCFC does. But that isn't an AJAX issue at all. It is just me reusing the same template. If your search returns N pages, the links show that I also allow passing the search term via the query string. So one change I could make would be to switch my FORM to a GET, although that's not technically valid. Either way - it isn't an AJAX issue though. Or are you talking about my _plans_ to add more AJAX? Or just AJAX driven search results in general?

Comment 20 by Charlie Arehart posted on 7/22/2007 at 11:18 PM

Well, I think you may be reading more into this than I meant. I'm not referring to any previous discussions we've had about BlogCFC and bookmarking search results.

I simply meant that many who build such Ajax apps may create their CFDIV (or whatever control) so that all the server interaction occurs within that control, and the user would never see a URL changing (much like when one uses an iframe).

I was just bringing that challenge to people's attention. In your case, I see that indeed the next/prev links cause a regular page request (where the URL changes to reflect it). I just hadn't noticed that, so one could look at that link to get the URL to bookmark.

But if you had instead designed those next/prev links to refresh the control holding the results instead, then this issue I mention would be something to note.

I wasn't attacking any choice you made. I just thought it interesting to note that some Ajax apps may indeed do new page requests without exposing a (changing) URL to the user, that's all.

Comment 21 by Raymond Camden posted on 7/23/2007 at 6:08 AM

No no - I didn't take it as an attack. I just didn't get WHERE you were coming from in relation to what I said. You are _definitely_ right.

Comment 22 by Charlie Arehart posted on 7/23/2007 at 7:14 PM

"We cool". :-)

Comment 23 by Neil Middleton posted on 7/24/2007 at 7:29 PM


Out of interest, how are you finding CFFEED. Feed-Squirrel is centered around home-grown RSS parser, and I'm wondering if it's worth the switch when the server moves to CF8.


Comment 24 by Raymond Camden posted on 7/24/2007 at 7:34 PM

In general - I like it. But I have some problems with the way queries are made. If you look at Paragator, one of the things I do is copy values over into common columns. That way I can always use one column for the link, for example, no matter what feed type it is. In my humble opinion, CFFEED should have done this for me. Now _normally_ I think people will use hard coded FEED urls and won't be doing it dynamically like I am.

THere is also a bug with some feeds not reporting dates correctly. But in the 290 or so feeds I aggregate, this was a small minority.

All in all - I'm VERY happy it was added, I'd just tweak it a bit for CF9. :)

Comment 25 by Neil Middleton posted on 7/24/2007 at 7:38 PM

Thats good to hear, I vividly remember the time I spent trying to write the parser for the FS core.