The use of locking in ColdFusion still appears to vex people. I thought it would be nice to write up a quick explanation as to why (and how) you would use locking in regards to file operations. Tomorrow (ok, maybe later in the week) I'll follow up with another blog post talking about locking in terms of data.
Let's begin with a simple example. Imagine you want to keep record of how many times people visit your web page. Tools like Google Analytics being too difficult to use, you decide to simply record the number of hits in a text file. (Just as a quick tip - don't do this. Please.) Your code then may look like this:
<cfif fileExists(fileName)>
<cfset contents = fileRead(fileName)>
<cfif not isNumeric(contents)>
<cfset contents = 0>
</cfif>
<cfelse>
<cfset contents = 0>
</cfif> <cfset contents++> <cfset fileWrite(fileName, contents)> <cfoutput>
Done. There are now #contents# views.
</cfoutput>
<cfset fileName = expandPath("./counter.txt")>
All this code does is define a file name (counter.txt) and if it exists, reads it in. If it doesn't exist (or it did and had some incorrect value), it defaults the value of contents to 0. Then we simply add one to it and write it out. Simple, right?
Now I want to imagine what happens if 2 or more users visit your web page at the same time. Remember that ColdFusion can serve the same file up to multiple people at once. If it couldn't, it wouldn't be able to handle load very well, right? Imagine 2 users hitting this file at the exact same time. ColdFusion reads in the file for both users at the exact same time. That means for both uses, the value of contents it the same number, let's say 1. Then ColdFusion carries on - adding one and writing it out - leaving a value of 2 when it should have been three. Let's see how locking can help fix this.
<cflock name="counterFileRead" type="exclusive" timeout="30">
<cfif fileExists(fileName)>
<cfset contents = fileRead(fileName)>
<cfif not isNumeric(contents)>
<cfset contents = 0>
</cfif>
<cfelse>
<cfset contents = 0>
</cfif> <cfset contents++> <cfset fileWrite(fileName, contents)>
</cflock> <cfoutput>
Done. There are now #contents# views.
</cfoutput>
<cfset fileName = expandPath("./counter.txt")>
I've added 2 lines here - an opening and closing cflock. Because I'm both reading and writing a value, I used an exclusive lock. This means that for this one block of the file ColdFusion will only allow one person to run it. So if we go back to our imaginary situation of two (or more) users hitting this file, they would both be able to execute the first line of code, but when they hit the lock, ColdFusion would handle making one wait while the other carries on. If our initial value was 1, then the first user would write out 2 in the file, and when the next user was allowed in, he would see 2 and write it out as 3.
The name is inconsequential, but should be something sensible. "counterFileRead" should be unique to the server. If anyone else used the same lock for some other purpose, you could have people waiting when they don't need to. If you plan on doing file read and writes of dynamic files, it would then make sense to make the lock name include the file name. Lock names can be any string, so by appending the file, I can make the template a bit more friendly if the fileName value was dynamic.
<cflock name="counterFileRead: #fileName#" type="exclusive" timeout="30">
<cfif fileExists(fileName)>
<cfset contents = fileRead(fileName)>
<cfif not isNumeric(contents)>
<cfset contents = 0>
</cfif>
<cfelse>
<cfset contents = 0>
</cfif> <cfset contents++> <cfset fileWrite(fileName, contents)>
</cflock> <cfoutput>
Done. There are now #contents# views.
</cfoutput>
<cfset fileName = expandPath("./counter.txt")>
You may ask - where would I use exclusive versus a read only lock? In my script, I'm reading and writing. Therefore I need an exclusive lock. We could modify the script a bit to not be a page view counter but simply a human counter. Ie, record every time a new person comes to visit:
<cfif not isDefined("cookie.firsttime")>
<cflock name="counterFileRead: #fileName#" type="exclusive" timeout="30">
<cfif fileExists(fileName)>
<cfset contents = fileRead(fileName)>
<cfif not isNumeric(contents)>
<cfset contents = 0>
</cfif>
<cfelse>
<cfset contents = 0>
</cfif> <cfset contents++> <cfset fileWrite(fileName, contents)>
</cflock>
<cfcookie name="firsttime" expires="never">
<cfelse>
<cflock name="counterFileRead: #fileName#" type="readOnly" timeout="30">
<cfif fileExists(fileName)>
<cfset contents = fileRead(fileName)>
<cfif not isNumeric(contents)>
<cfset contents = 0>
</cfif>
<cfelse>
<cfset contents = 0>
</cfif>
</cflock> </cfif>
<cfoutput>
Done. There are now #contents# visitors.
</cfoutput>
<cfset fileName = expandPath("./counter.txt")>
In this version we check for the existence of a cookie. If it doesn't exist, we proceed as before. Lock, read, increment, and write. If the cookie does exist, we just read. Notice that the lock name remains the same. That's crucial. However we can switch to a readOnly lock for that operation as we aren't modifying the value. (To be honest, we could probably get rid of the lock completely. If the value actually changed between when we read it and stored the value, it probably wouldn't matter in the real world.)
Archived Comments
Great article. Never really got my head around read only. Actually, I don't mind admitting that locking is something I have never fully got my head around at all, and being long time user of Sandra Clark's Fusebox 'request' plugin, that writes the Session scope into the Request scope and back into the Session scope with each page request, I have not had to worry about it too much. Incidentally, in the case of a cflocation etc, she also has a script called 'fnc_writerequest.cfm' to write back into the Session scope before the end of a page request. I do rely on cftransaction on one of my sites that sometimes process many transactions per second though - http://sms2screen.com.au. In particular though, I have never got my head around not locking Session variables around some database queries, and remain a little puzzled with Sean Corfield's 'underscore' (on line two) of my customised version of one of his locking examples (below). Would love to know about that, although the double struct check does make sense.
<cfif NOT StructKeyExists(Application, "Security")>
<cflock name="#Application.applicationName#_Security" type="exclusive" timeout="10">
<cfif NOT StructKeyExists(Application, "Security")>
<cfset Application.Security = CreateObject("component", "model.security.Security").init(Request.dsn) />
</cfif>
</cflock>
</cfif>
I forgot to mention. It's handy to know that the lock name can be any string, I didn't know that. One thing that I have never been able to work out though, is how long to set timeout. What should that be based on?
When you say you don't understand the underscore, do you mean in the lock name? All he did there was create a name based on the current application name (that's a built in variable) followed by _security. So given an app called beer, the lock would be beer_security. If you used the same code in another application, wine, it would be wine_security. The end result is an application specific named lock.
Timeout should be based on the reasonable expectation of how long you think a process would take and your traffic. Given that a fileWrite is probably going to take 100ms or so, if I had 100 people hit my site at the same time, then it would take 10 seconds total for them to all get an exclusive lock and write out. 30 seems safe in that regard then.
D'oh! I should have picked the underscore issue up from where you mentioned the lock name can be any string. Makes sense now, as well as the calculation for timeout. Thanks!
Is there any reason not to just append a 4-5 digit random number on the end of the lock name? (eg, "SecLock_#RandRange(1, 10000)#") Seems like it takes the time out of coming up with names, or worrying your convention is too conventional and is being repeated.
Ovwerall, I wonder why locks aren't just unique by default. It seems bug-prone and ineffcient to rely on users to manage this, versus just handling it programatically.
Paul, making the lock random would totally defeat the purpose of the lock. When 2 people hit the page, they would have different lock names, and the code block would NOT be single threaded.
As to why this isn't automatic - it's one of those things. A compiler simply can't be "smart" about this. It requires a human to say - this is an operation that I need to single thread. You may - for example - not care. Consider the hit counter. You may decide that you NEVER want to slow people down, and if the hit counter is off by a few points, that's ok.
The one thing I would be careful of is doing file locks when reading files via unc pathing. I have seen, from personal experience, that the lock can get stuck on the file.
For example, lets day you put a exclusive read lock on a file. In the process of reading that file the network goes down. ColdFusion never finished its read so it was never able to release the lock. The file is now stuck in a perpetual lock state.
--Dave
Interesting. Does CF crash or does it continue to serve _other_ requests and only requests using that lock get hung?
We have an app that lets users download files from the server using the standard combination cfheader and cfcontent to accomplish the download. Should the that code be cflock-ed since there is a chance that several users may try to download the same file at the same time or is this different?
Ask yourself - does it matter if N people download the file? Probably not. You aren't modifying the file - simply serving it.
@ray From what I have seen it just fails that thread. You end up getting a file read error. Other threads continue just fine and CF stays running.
Ray,
Don't forget to double-check your conditions if you're conditionally locking code. For example, this:
<cfif not isDefined("cookie.firsttime")>
<cflock name="counterFileRead: #fileName#" type="exclusive" timeout="30">
<cfif fileExists(fileName)>
etc.....
Should be this:
<cfif not isDefined("cookie.firsttime")>
<cflock name="counterFileRead: #fileName#" type="exclusive" timeout="30">
<cfif not isDefined("cookie.firsttime")>
<cfif fileExists(fileName)>
etc.....
Otherwise you wind up with a potential race condition under heavy load. Even though it's a small possibility, two requests *could* hit that first "if" statement and try to lock at the same time - if they do, you don't want the code inside of the locked section to execute twice.
That would only occur with two requests from the same user. Possible with frames, or tabs, but highly unlikely. Still - good point.
Or AJAX requests - I think it's actually even more important in the context of AJAX.
Good Article Ray!
But I always consider doing the Lock Timing as Timestamps and Store them In DB and Fetch them, I do not trust cflock.
Various Reasons!
1. What happens if you shut the browser from the Taskbar!
2. What Happens if the Browser gets Closed Unexpectly.
3. Power Failure anotehr Reason.
and Others many i do not remember yet!
@Misty: None of the things you mentioned mattered. Given our example of 2 users hitting the file at once, if user's 1 browser closes, CF still continues the request and the lock end.