Building a simple ColdFusion Token/Template System

This post is more than 2 years old.

As ColdFusion developers, we build a lot of different types of applications. For the most part though these apps come down to simple content management systems. We build out forms and let our clients enter blocks of text that then get put together nicely on the front end. While the actual view of the application is dynamic, typically the text itself is rather static. So for example, this blog post comprises a title and a body, and is certainly dynamic when viewed by the public, but the actual words themselves are just straight text. Sometimes we need to provide a way to add an additional level of dynamicness (yes, I know that isn't a word, but I'm going for it anyway). A great example of this is an email template. Your web site may send out emails to users on a weekly schedule. The text of the email may need to include customization. It may want to say something like, "Hello #name#", where name is the person receiving the email. You can do this quite easily in ColdFusion, but what if the client wants more control over the text? What if they want to change "Hello" to "Hiya" or perhaps add a ! after the name? This is where a simple token/template system can come in handy. Here is a quick UDF I built with a few sample uses of it. (And by the way, I'm pretty sure I've blogged about this before in the past few years, but it was on my mind this week so I thought I'd whip out another example.)

To begin - let's look at a simple template. You can imagine our client editing this to create a customize email for a product site. Users are allowed to sign up for specific newsletters based on different types of products. Here is the plain text:

Hello {name},

Thank you for signing up for our {newslettertype} news list. Every day we will send you new and exciting emails about {producttype}.

Thank you, {source}

I've decided on the use of { and } to wrap the dynamic aspects of the template. Any character could be used really but you want something that stands out from the rest of the text. Now that we've got a block of text, we need to create a ColdFusion UDF that will:

  • Look for and find all the {...} strings in the text
  • Accept data in that contains the "real" values for those tokens
  • Replace the tokens with real data and return the string

Here is the UDF I came up with. It is a bit complex so I'll go into the design decisions after the paste.

<cffunction name="tokenReplace" output="false" returnType="array"> <cfargument name="string" type="string" required="true" hint="String with tokens inside, specified by wrapping with {}"> <cfargument name="data" type="any" required="true" hint="Data used to replace tokens. Can be an array of structs, a struct, or a query object.">

&lt;cfset var result = []&gt;

&lt;!--- first, find tokens in the string so we know what we can recognize ---&gt;
&lt;!--- note that we will use a list, which means we wouldn't support a token with a comma in it, which is fair I think ---&gt;
&lt;cfset var knownTokens = ""&gt;
&lt;cfset var matches = reMatch("{.+?}", arguments.string)&gt;
&lt;cfset var match = ""&gt;

&lt;cfloop index="match" array="#matches#"&gt;
	&lt;cfset knownTokens = listAppend(knownTokens, replaceList(match, "{,}",""))&gt;

&lt;!--- based on our data, do different things ---&gt;
&lt;cfif isStruct(;
	&lt;cfset var thisResult = arguments.string&gt;
	&lt;cfloop index="token" list="#knownTokens#"&gt;
		&lt;cfset thisResult = rereplace(thisResult, "{" & token & "}",[token], "all")&gt;
	&lt;cfset arrayAppend(result, thisResult)&gt;
&lt;cfelseif isArray(;
	&lt;cfloop index="item" array=""&gt;
		&lt;cfset var thisResult = arguments.string&gt;
		&lt;cfloop index="token" list="#knownTokens#"&gt;
			&lt;cfset thisResult = rereplace(thisResult, "{" & token & "}", item[token], "all")&gt;
		&lt;cfset arrayAppend(result, thisResult)&gt;

&lt;cfelseif isQuery(;
	&lt;cfloop query=""&gt;
		&lt;cfset var thisResult = arguments.string&gt;
		&lt;cfloop index="token" list="#knownTokens#"&gt;
			&lt;cfset thisResult = rereplace(thisResult, "{" & token & "}",[token][currentRow], "all")&gt;
		&lt;cfset arrayAppend(result, thisResult)&gt;
	&lt;cfthrow message="tokenReplace: data argument must be either a struct, array of structs, or query"&gt;

&lt;cfreturn result&gt;


So the first argument to my UDF, tokenReplace, is the actual string. That should be easy enough to understand. The second argument is a bit more complex. I wanted a system that would allow me to pass different types of data. I imagined a query would be most often used, but I also imagined you may need a "one off" so I also wanted to support structs. I also figured you may have an array of data as well so I added support for arrays of structs. Since I figured you would probably be working with multiple sets of data more often than not, I made the UDF consistently return an array of strings.

I begin by getting all my tokens. This is done with (edited a bit):

<cfset matches = reMatch("{.+?}", arguments.string)>

<cfloop index="match" array="#matches#"> <cfset knownTokens = listAppend(knownTokens, replaceList(match, "{,}",""))> </cfloop>

The reMatch finds all the {...} tokens and the loop removes the {} from them. Once I have that, I then split into a branch that handles my three types of data. In all cases though we assume that if you have a token for foo, you have data for foo. I don't care if you send me more data than I need, but I do care if you don't have data for one of my tokens. Right now if you make that mistake, you get an ugly error, but the UDF could be updated to make it more explicit.

So, let's look at a complete example:

<!--- Our template ---> <cfsavecontent variable="template"> Hello {name},

Thank you for signing up for our {newslettertype} news list. Every day we will send you new and exciting emails about {producttype}.

Thank you, {source} </cfsavecontent>

<!--- Our sample data ---> <cfset data = queryNew("id,name,newslettertype,producttype,source","cf_sql_integer,cf_sql_varchar,cf_sql_varchar,cf_sql_varchar,cf_sql_varchar")> <cfset queryAddRow(data)> <cfset querySetCell(data, "id", 1)> <cfset querySetCell(data, "name", "Bob")> <cfset querySetCell(data, "newslettertype", "Weapons of Mass Confusion")> <cfset querySetCell(data, "producttype", "weapons")> <cfset querySetCell(data, "source", "MADD")>

<cfset queryAddRow(data)> <cfset querySetCell(data, "id", 2)> <cfset querySetCell(data, "name", "Mary")> <cfset querySetCell(data, "newslettertype", "Death Rays and MegaSharks")> <cfset querySetCell(data, "producttype", "lasers")> <cfset querySetCell(data, "source", "Dr. No")>

<cfset queryAddRow(data)> <cfset querySetCell(data, "id", 3)> <cfset querySetCell(data, "name", "Joe")> <cfset querySetCell(data, "newslettertype", "Good Meat to Eat")> <cfset querySetCell(data, "producttype", "food")> <cfset querySetCell(data, "source", "PETA")>

<cfset results = tokenReplace(template,data)>

<cfdump var="#results#" label="Query Test"> <p/>

<cfset s = {name="Luke", newslettertype="Lightsabers", producttype="swords", source="The Empire"}> <cfset results = tokenReplace(template, s)> <cfdump var="#results#" label="Struct Test"> <p/>

<cfset s2 = {name="Scott", newslettertype="Beers", producttype="beer", source="The Beer Industry"}> <cfset array = [s,s2]> <cfset results = tokenReplace(template, array)> <cfdump var="#results#" label="Array Test">

As you can see, I've got my template on top, and then three sets of examples. One query, one struct, and one array of structs. The results show that in all cases, we get nicely customized messages back.

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA

Archived Comments

Comment 1 by PaulH posted on 11/2/2010 at 9:00 PM

been there, done that ;-)

the icu4j messageFormat makes this sort of thing very easy. see it in action here:

Comment 2 by Stephen Moretti posted on 11/2/2010 at 9:00 PM

We have some scary rendering system that works off of a tree of resources that you build up from an understanding of what positions and qualifiers (tokens) will be required in a page which you then apply to a template.

At the end of the day it boils down to a massive version of your search and replace function :)

Don't get me wrong, we need all of the extra application to make building pages with multiple applications and applications with multiple relationships to other applications possible, but in the end its just replaces place holders with content.

Comment 3 by Raymond Camden posted on 11/2/2010 at 9:02 PM

@Paul: Heh, I never said it _hadn't_ been done - I even said I probably did it here before. ;)

Comment 4 by Kevin Marino posted on 11/2/2010 at 9:02 PM

Nice to see this brought up again. I often have to explain token/ templating when dealing with dynamic pdf generation or email templates and being able to point to someone elses work is often helpful.

Comment 5 by Tony Nelson posted on 11/2/2010 at 9:25 PM

It would be cool if we had a server-side ColdFusion templating system that supported the same markup as a client-side templating system, such as Handlebars.js, mustache.js, or the jQuery Templates plugin.

Comment 6 by Tony Nelson posted on 11/2/2010 at 9:26 PM

After a quick search, it looks like Mustache has already been ported to ColdFusion and even listed on the Mustache home page:

Comment 7 by PaulH posted on 11/2/2010 at 9:28 PM

@ray well i had thought you'd forgotten, what with all that appearing in true blood & all ;-)

seriously though the MessageFormat class is slick, handles formatting, calendars, etc for you across all locales. it's the bee's knees. there's one in core java if icu4j is overkill.

Comment 8 by Henry posted on 11/2/2010 at 10:45 PM

FYI I think there's a token/template library in cfcommons as well.

Comment 9 by Kevin Hall posted on 11/3/2010 at 6:55 PM

Thanks for this, I had this exact problem coming up on my to-do list. I have a database table of phrases in multiple languages but needed to insert names and other data in certain places for things like welcome and thank you messages. This is perfect for that task. Once again you solve my problem a few days before I need to.

Comment 10 by Tyler Clendenin posted on 11/4/2010 at 8:55 AM

I actually went with Freemarker for this. Freemarker is incredibly powerful.


Docs: http://freemarker.sourcefor...

Here is a blog post that I wrote with how I did it with an example

Please let me know if I did anything wrong/bad.

Comment 11 by Misty posted on 11/9/2010 at 2:11 PM

Thanks Ray! It means we can really the email templates or any kind of template in a database and then later use it in our cfc to send mails.

Please correct me if i am wrong!

Comment 12 by Raymond Camden posted on 11/9/2010 at 5:50 PM

Yep, you can.

Comment 13 by Tom posted on 10/23/2012 at 8:49 PM

What version of CF does this need to be on? On CF8, I'm getting an error:

Local variable thisResult on line 19 must be grouped at the top of the function body.

The error is referencing in the UDF, <cfset var thisResult = arguments.string>.

Comment 14 by Raymond Camden posted on 10/23/2012 at 9:01 PM

CF9 added the ability to use VAR statements anywhere. For CF8 support, move that to the beginning of the method.

Comment 15 by Tom posted on 10/23/2012 at 11:20 PM

I see what I was doing wrong - your udf is ok as-is. I was calling the wrong table in my database. Thanks for the help on this!

Comment 16 by Tom posted on 10/23/2012 at 11:31 PM

What if I wanted to construct a database of replacement terms. Using your example, a DB query would look something like this (had you had an actual datasource)
SELECT name, newslettertype, producttype, source
FROM replacementText
But if I wanted a list of terms and its replacement text that I could use in other places, too (e.g.:
SELECT term, replacementterm
FROM replacementText

I hope the formatting is readable after I click Post. :)

Comment 17 by Raymond Camden posted on 10/24/2012 at 1:19 AM

Sorry - I don't quite get what you are asking Tom. Can you rephrase it?