An About Me page. It doesn't matter if you don't want to name yourself. You should have a very defined, very clear page that either says who you are or why you are not sharing your name. And frankly, unless you are a political blogger in a country actively shooting dissidents, your reason to be anonymous is probably crap.
A Contact page. You don't (and probably shouldn't) give out your email address. But provide a way for me to contact you. If I find a bug on your blog, or want to hire you, or whatever, I don't want to leave a comment on a random blog entry.
A way to subscribe to comments. I hate it when I leave a comment someplace and discover I have to refresh the view to see if there are any follow-ups. I'm almost to the point where I won't bother commenting if you are going to make me work to see replies.
A way to subscribe to entries. This one I'm willing to be forgiven on. There are multiple services that will monitor RSS feeds for you. But it's convenient if the blog itself offers this.
Oh yeah - and a RSS feed. And don't hide it. You should use the META tag browsers pick up on as well as a very nice and obvious RSS button/link/etc.
All of the above should be quick and easy with any half decent blog engine.
Written in three minutes entirely off the cuff so let the flaming begin... ;)