A while back I posted about how I had begun to see Node in a new light. (Do folks refer to it as "just" Node or do you always include the JS at the end?) I had some time this week and decided to try building a real, if simple, application. Since I had plenty of data for my blog, I thought a simple blog viewer would be a good application to try building. Here are some random, scattered thoughts on what the process was like.
First and foremost - I had decided to use Express to build a web application. Express is a Node application framework specifically built for web applications. I had seen Sim Bateman demo this at cfObjective this year and it had looked pretty cool. What I liked right away is how I had complete control over how requests were handled. I was easily able to specify a folder for static crap, like CSS and JavaScript. I then could specify URLs and how to handle them. In ways this is a bit like ColdFusion's onRequestStart, but at a much deeper level. You get the ability to say, for example, when a request comes in for /ray/bio, run so and so code. But you can also easily add dynamic patterns as well. If I wanted to match /display/X where X was dynamic, I could set up the code easy enough and Express would automatically give me access to the dynamic portion as a variable. (Before I go any further - let me point out that I may confuse what Express gives you versus what is just plain there in Node. Please forgive me if I make that mistake.)
Here is an example of all three above. From what I know, the patterns are checked in order of how I coded them.
As I said, the first block handles my static stuff. I've specified a folder called public and inside of there I'd place my CSS and JS files. Once I drop in a style file I'd address it by just pointing the URL to /public/style.css.
The second block handles the home page request. Don't worry about "blog" just yet, I'll explain it in a bit. But basically, you can see that I'm calling for some data and then rendering it. (Again, more on that in a second.)
Finally - I added support for loading one particular blog entry. Note that I was able to precisely define a URL pattern of /entry/X. I could have done anything here at all. I love that freedom. To be clear, you can do the exact same in ColdFusion if you add in a URL rewriter like what's built into Apache. But I like having it right here in my application. It makes it a bit easier to mentally grasp what is going on in the application.
If you visit my blog often, you know that I use a slightly slicker URL scheme. It took me then 5 minutes to add this to my application:
Ok, so that's some examples of responding to URLs. I've got more coming up, but let's talk about something else - rendering the pages. I love JavaScript, but there is no way in heck I'm going to build out HTML views in JavaScript. As much as possible I've been trying to use templating engines lately. As I was researching, I discovered that a library called EJS worked with Express. Hooking it up to Express was rather simple. I told my app I needed EJS and told Express to use it to parse HTML files. Honestly I don't 100% understand this portion, but it worked:
If you look at the app.get("/") block above, you can see where I run res.render(). The first argument is the name of a template to run. By default this will be in a subdirectory called views. The second argument is data I'm passing to the view. Here is what home.html looks like - and remember - I did this very fast and kinda ugly. Normally you would have a bit more HTML in there:
I am not a fan of the template syntax. Frankly it felt like I was writing classic ASP. That being said - it did work. Note that I'm doing a bit of work to create the fancy URL. EJS supports (I believe) writing your own helper functions. Normally I'd have built something to simplify that so that the view had much less logic in it. Just assume that as my first view I didn't write this as nicely as I would if given more time. As a comparison, here is the view for an individual blog entry:
As I mentioned, I'm not really a fan of EJS. There are other alternatives. Right now I'm considering Dust, but as I ran into problems with that, I couldn't actually use it.
Hopefully at this point you have a rough feel for how Express lets you handle requests and specify views to actually render them. Let's talk about the database layer. After I figured out how to do my views and pass parameters around, I needed to get database support. This is where I got to play around more with NPM. NPM, or the Node Package Manager, is an incredibly powerful tool and probably one of the main reasons Node is so popular. (Other platforms have similar support.) From the command line you can tell Node to get a package (think open source project focused on adding a particular feature) and install it to your system. You can also tell your application itself that it requires a package. So for example, my application needs Express and MySQL support. I can use a special file (package.json) to note these requirements, run one command, and all the supporting libraries just magically come in.
But... this isn't all rainbows and unicorns. When I decided to add RSS support to my application, I used NPM to search for an RSS library. If I remember right, about 30 or so packages showed up. I froze like a deer in headlights. Don't get me wrong, I like options, but I had absolutely no idea which one to pick. This is very much like the problem you may have with jQuery plugins. You can almost always count on jQuery having a plugin to do X, but finding out what the "best" one is can be a laborious process. I feel like I'm going to get some criticism on this, but I do wish people would keep this in mind when praising Node. For me, I picked the package with the name "rss" just because it had the simplest name. Luckily, the one I chose worked great. I was able to add RSS support soon after:
But it didn't always work out well. I mentioned above that I tried to use another templating engine. The first one I tried, Dust, didn't work because it wasn't supported on Windows. That really surprised me. Shouldn't JavaScript work everywhere? To be fair, the reason the project wasn't supported on Windows was because the author didn't have an environment to test with (and he did the right thing then in marking it not supported), but I ended up getting stuck for a while.
So going back to database support, it turns out there are a few options for working with MySQL. Unfortunately, none of them really instilled a great deal of confidence in me. In fact, the solution I went with didn't even support bound parameters! Yes, you could use them in your code, and yes, your data would be escaped, but it wasn't truly using a bound parameter in its communication to the database. And frankly - as much as I like JavaScript, I'm not sure how much I'd trust a database library written in it. I haven't done any performance tests, but out of everything I did, this was the one area that gave me the most doubt.
With all that being said, using MySQL was pretty easy. I began by just setting up the connection like so:
I then created a module called blog that would handle my service layer. Since I had a "con" object that represented my database connection, I exposed an API where I could pass it in to the blog:
Here then is my blog module. A 'real' blog engine would have quite a bit more of course but you get the idea.
So what do I think? I love it. Once I got my environment running (and be sure to use nodemon to make reloading automatic) I was able to rapidly build out a simple application. I loved the level of control I had over the request and how quick it was to get up and running. I didn't love the fact that the quality wasn't quite consistent across various modules.
p.s. One more code snippet. I demonstrated the RSS support above. But I also built in a quick JSON view as well. It was incredibly difficult. Honest. This took me hours to write. (Heh...)
Archived Comments
Express also supports Jade as a template engine. Was there something about it that didn't work for you too? I am just trying to get started with Node/Express and if there is some reason why Jade didn't work for you, it would be useful to know. Also, did you give any consideration to MongoDB? It seems like it would be a good fit for something as document-centric as a blog.
Great entry.
My team is currently in the process of switching over our Coldfusion site to the NodeJS platform, it's quite the experience!
Express is the framework we also have chosen (it's great and there's really no better choice for a web app).
As for a templating engine, we're using Jade (http://jade-lang.com/), which is working well (our designer LOVES it). With a little bit of tweaking we were able to get it functioning with our Backbone based front-end as well which is extremely helpful.
I think a good post would be an example of how to use package.json and npm to setup your dependencies.
Finally, here are a couple of great modules we're been using (Listing them by their name in npm)
async - Great library for keeping yourself out of "callback hell". As you dive deeper into Node, you'll find yourself loving this one IMHO.
config - Very simple module for setting up easy config files, supports js, json or yaml configs
node-cache - Quick and simple node in memory caching solution
mongoose - I believe this is the goto MongoDB driver. Like you mentioned with MySQL and RSS, there are a bunch of choices but this one appears to be the most robust.
Small side note. For the json viewer part.
With express you can just use:
res.json(rows);
@Rob: I absolutely despise Jade. It is a purely personal thing. Therefore I didn't even try. (To be clear, I hate the syntax, not the project. You get what I mean.)
MongoDB: The reason I didn't use it was because my datasource already existed. It's the same one I use for this blog (well, a local copy).
@Danny: re: package.json - I only "kinda" get it which is why I didn't spend a lot of time on it here in the post. Actually, I had difficulty finding documentation for it! (Although I didn't spend a lot of time.) I get the basics of it, and appreciate its power.
re: async: I'll have to check it out. I was thinking maybe I'd just use jQuery and Deferreds, but a) it seems a bit wasteful to load all of jQuery just for that and b) I'm not even sure if it can run server side.
config: It seemed like Express had good config stuff built in. Was it not good enough?
node-cache: Cool, I'll have to check it out.
res.json: Any reason why I'd use that over JSON.stringify? Sure it is a bit smaller - but it seems so simple I'm not sure why I'd use a wrapper?
Using res.json is purely a preference thing but it allows you to skip res.end() and res.writeHead() res.json automatically sets the content-type to application/json and you can also specify the status code in it:
IE res.json(500,{errorMessage: 'oh noes!'});
As for config, we just found it to be a very clean way for setting up defaults and different configs for development/production servers.
I.E. Load dev.yaml on server1 and prod.yaml on server2, then require('config').defaults.path_to_solr
We didn't quite find that level built right into express, but it's possible we didn't dig deep enough.
Ah - ok - that sounds like a perfect reason to use that instead. Thanks Danny.
On the subject of jQuery server. Yes, it does run serverside in Node and you can install it via NPM.
You're right that it's pretty wasteful though because the bulk of jQuery is the selectors and DOM manipulation.
We've found that underscore (coupled with underscore.string sometimes) on the serverside is a stronger choice.
I wish I could get express to install on Mac Lion, can't get it to work using 'npm install express'....
Could really use someone's help, there's chatter of forums that NPM may be down at the moment....
Check the error. When I did it in OSX, the error said to run as Admin. When I did sudo npm install express, it worked fine.
I've been toying through the tutorials of node. I saw you posted this.
Is there any chance you have the entire project available for share anywhere?
I know all the modules are downloadable through npm but I'm interested in seeing the structure and layout for something more advanced then "hello world".
Also, have you done anything with MSSQL? I've been browsing around and everything seems to be half working, abandoned, and the microsoft seems to be dependant on having 5 other things installed and Node running on windows!
Steve: I could share my blog demo. It wouldn't be terribly useful w/o the database, but if it would help, I'd share it.
And no - I haven't done anything with Node and MSSQL.
Thanks for the reply.
I've been searching and coming across a lot of blogs this morning that have pushed me closer to the right direction. I installed mysql and have figured out how to query one of their sample db's, sending variables, etc....
I'm also trying to figure out how you incorporate templating like jade or ejs, but I am reading that there are commands that generate a skeleton app for you to get started with.
every DB thing I've ever done in CF has been on sql 7-2008, so my hopes were to tie into one of those. but I think until I get the rest down for now, I'll play around with th mysql db's.