Rob sent in an interesting question this morning concerning tracking users and their activities. It is a rather long read, but please check it out and let me know what you think. (Also, I've been getting a lot of email from Australia lately. I knew there were a bunch of ColdFusion developers down there, but who knew there were so many! I'm going to have to buy some Coopers and fly down there some time.)
First - let me paste in his question:
I've been trying to figure out is a tracking feature that allows me to track the users around the application, I'm sure you can imagine the benefits of knowing what the little monsters have been up to when you're not around.This tracker should have two sides to it, firstly a 'track' for each user in the application, it logs everything they do around the app and when they do it into an array or something or that nature, then when the session ends or they log out of the application, this data will be passed into persistence.
The second element of the tracker is from an application admin point of view. I'd love to have a list of users that are currently active on the application, what their current activity is and suchlike.
Both of these elements are fairly simple to implement on their own, but I'm after your thoughts on where they should be stored and whether they should be contained within a single cfc, as they essemtialy perform very similar tasks keeping them as two separate components feels a little bit like over kill.
As I see it we could either place the first cfc which stores all the users activities into their session scope, and the cfc that lists current users on the system into the application scope. Then when a user does an activity, we add it to the first cfc and update the second. The second option is to keep all the core function in a single component in the application scope, we then have a structure, each key representing a user and the value being an array of all their activities. We then place an identifier for their track into their session scope, so when they log out or the session ends, we can identity their track in the application scope and move it to persistence.
So first off - before we discuss if this should be one CFC or two (and bear in mind, this is all about opinion, either solution will work), you may want to ask yourself if storing it in CFC memory (RAM) is actually what you really want. While it certainly makes it easy to get at, you aren't going to have any history. You will be able to see what people are doing now - or even through the lifetime of your Application scope, but what about two weeks ago? Two months ago? I'd first ask if you shouldn't be doing simple database logging instead. This would give you wonderful historical data. You may be implying that by your "move it into persistence" comment at the end. But this would worry me (by this I mean doing it at the end). If the server crashes, then you will have lost your current data.
If you do decide to log - the catch to this is what you should log. You mentioned you wanted to log their actions. So lets say you log something like this to the database:
User Bob visited page: Jobs
And then later on you modify this to be:
User Bob hit page: Jobs/Main
Suddenly you have a data integrity issue as the two records won't match up. This issue is surmountable though - just spend some time really nailing down how you will log and try to ensure it will be forwards compatible with future changes.
Now lets ignore that. Because whether you log to files or use RAM, your question still applies. I definitely agree that one CFC in the Application scope makes the most sense here. I try to keep my CFCs are organized as possible, and this "User Activity" function seems to call for it's own CFC.
For fun - I decided to whip up a demo. Consider the below CFC:
<cfcomponent output="false">
<cffunction name="init" returnType="usermonitor" access="public" output="false">
<cfset variables.users = structNew()>
</cffunction>
<cffunction name="addUserActivity" returnType="void" access="public" output="false"
hint="Adds a user activity.">
<cfargument name="key" type="string" required="true">
<cfargument name="activity" type="string" required="true">
<cfif structKeyExists(variables.users, arguments.key)>
<cfset arrayAppend(variables.users[arguments.key].activities, arguments.activity)>
</cfif>
</cffunction>
<cffunction name="getCurrentActivity" returnType="struct" access="public" output="false"
hint="For all users, return the latest activity.">
<cfset var result = structNew()>
<cfset var user = "">
<cfloop item="user" collection="#variables.users#">
<cfset result[user] = variables.users[user].activities[arrayLen(variables.users[user].activities)]>
</cfloop>
<cfreturn result>
</cffunction>
<cffunction name="getUserActivity" returnType="array" access="public" output="false"
hint="Gets a user activity history.">
<cfargument name="key" type="string" required="true">
<cfif structKeyExists(variables.users, arguments.key)>
<cfreturn variables.users[arguments.key].activities>
<cfelse>
<cfthrow message="Invalid User">
</cfif>
</cffunction>
<cffunction name="getUserCount" returnType="numeric" access="public" output="false"
hint="Returns the number of users in our list.">
<cfreturn structCount(variables.users)>
</cffunction>
<cffunction name="getUsers" returnType="string" access="public" output="false"
hint="Returns the user list.">
<cfreturn structKeyList(variables.users)>
</cffunction>
<cffunction name="registerUser" returnType="void" access="public" output="false"
hint="Adds a user to our list.">
<cfargument name="key" type="string" required="true">
<cfset variables.users[arguments.key] = structNew()>
<cfset variables.users[arguments.key].data = structNew()>
<cfset variables.users[arguments.key].activities = arrayNew(1)>
</cffunction>
<cffunction name="unregisterUser" returnType="void" access="public" output="false"
hint="Removes a user from our list.">
<cfargument name="key" type="string" required="true">
<cfset structDelete(variables.users, arguments.key)>
</cffunction>
</cfcomponent>
This CFC works much as you described it - storing users and activities. I added some nice convenience functions as well. So not only can you get users, you can get a simple count as well. The function getCurrentActivity will give you the last activity of all users, which is a nice way to get a snapshot of their activities. I updated my Application.cfc to create this CFC, and used onSessionStart/End to register/unregister my user:
<cfcomponent output="false">
<cfset this.name = "um_test4">
<cfset this.applicationTimeout = createTimeSpan(0,2,0,0)>
<cfset this.sessionManagement = true>
<cffunction name="onApplicationStart" returnType="boolean" output="false">
<cfset application.usermonitor = createObject("component", "usermonitor")>
<cfreturn true>
</cffunction>
<cffunction name="onMissingTemplate" returnType="boolean" output="false">
<cfargument name="targetpage" type="string" required="true">
<!--- log it --->
<cflog file="missingfiles" text="#arguments.targetpage#">
<cflocation url="/apptest/404.cfm?f=#urlEncodedFormat(arguments.targetpage)#" addToken="false">
</cffunction>
<cffunction name="onRequestStart" returnType="boolean" output="false">
<cfargument name="thePage" type="string" required="true">
<cfset application.usermonitor.addUserActivity(session.urltoken,"Viewing page #thePage#")>
<cfreturn true>
</cffunction>
<cffunction name="onRequestEnd" returnType="void" output="false">
<cfargument name="thePage" type="string" required="true">
</cffunction>
<cffunction name="onError" returnType="void" output="false">
<cfargument name="exception" required="true">
<cfargument name="eventname" type="string" required="true">
<cfdump var="#arguments#"><cfabort>
</cffunction>
<cffunction name="onSessionStart" returnType="void" output="false">
<cfset application.usermonitor.registerUser(session.urltoken)>
</cffunction>
<cffunction name="onSessionEnd" returnType="void" output="false">
<cfargument name="sessionScope" type="struct" required="true">
<cfargument name="appScope" type="struct" required="false">
<cfset arguments.appScope.unregisterUser(arguments.sessionScope.urltoken)>
</cffunction>
</cfcomponent>
Lastly I built a simple little test page:
Current Users:
<cfdump var="#application.usermonitor.getUsers()#">
<p>
User Count:
<cfdump var="#application.usermonitor.getUserCount()#">
<p>
My Activity History:
<cfdump var="#application.usermonitor.getUserActivity(session.urltoken)#">
<p>
Current Actitivity:
<cfdump var="#application.usermonitor.getCurrentActivity()#">
I wrote all of this in 10 minutes, so do not consider it "tested" (nor more than say - oh - Windows), but if it helps any, enjoy.
Archived Comments
Great scott!!! :-o
That was a very fast and thorough response Ray, clearly using the force to bend the concept of time and reality as we know it! Thanks a great deal.
Your mock up looks very similar to the concepts that I've been playing with, almost TOO identical in fact *looks suspiciously at Ray.
Thanks for clarifying that it should be stored into the app scope as a single CFC, I'll agree that this method defiantly made more sense from an organisational point of view, i just had this nagging in the back of my mind that any user details should really be stored in the session where possible, but I think this application is a fair enough exception.
Now, the whole persistence issue; the concept that I was working on is that all the data should be stored in a struct/array as the user progresses around the application and then persisted to the database upon the completion of their session or when they log out.
Now I'll agree that this does eat up a bit of memory, but memory is cheap, and it may also be a problem if the server was to crash, but I favoured it a little over the standard database logging point of view from performance for the user, in this world of crazy cached queries and stored procs (got to love that in cf8) I think from my personal perspective I like it committing their activity at the end instead of live as it cuts down on the DB traffic, but either implementation would work a real charm I'm sure.
I suppose it’s like the old battle of the shopping cart, keeping it in a session array or query object does make more sense, but the database version would be the ‘safer’ option.
I look forward to hearing other opinions on this one, thanks again for your time Ray, you’re a stout scout! I'm off to polish this concept off and add a few other ideas once it’s done, I'll be sure to post you the final revision once I'm done.
Thanks,
Rob
Yeah, I have one that I dubbed "Carnivore" and it's pretty flexible / configurable. Can even be turned off without impacting the site and existing users that are logged in. On a high bandwidth site, it won't make sense to have it turned on, but it's nice to have the options.
Logging is a must though.
Heh, I thought I made it clear my code was definitely inspired by your descriptions. ;)
So I know RAM is cheap... but I still think a quick DB call would be ok performance wise. I'd maybe try it. Then again, losing a few sessions due to a server crash probably aint the end of the world either.
I have done something like this in the past. What I did was use the OnRequestEnd in application.cfc. In there I put an asynchronous call to a tracking cfc. This allowed me to track the user without slowing the user down. I was then able to use a file log that was then later processed into the database. I did it this way so I could bulk insert the data and not have tons of little inserts.
--Dave
I agree Ray,
I think that its definatly an issue that needs to be addressed somehow, I'll play around with the idea of sticking it directly in the database and see how it pans out against the array method, it would be nice to know it was fail safe against any server problems.
I like your method for using the URL key, the concept i'd worked on had the registeruser() method issue a UUID which represented the tracker, that was then placed in the session, but yours is much cleaner :-D
>> Heh, I thought I made it clear my code was definitely inspired by your descriptions. ;)
I was only pulling your leg when i said it looks suspiciously similar! :-D I guess great minds think a like!
Thanks again for your advice Ray.
I also like your concept Dave, sounds like it would be a nice way to stem the database traffic on pages that log multiple activities, where you might end up with 5 or 6 inserts on a single page, I like it.
Rob
Rob, that key is a part of the default session data set by all sessions in CF. It is unique so it is useful for stuff like this.
Thanks Ray, I'd seen the key before when dumping my session scope but never really thought of using it like this, I can think of a few other places in the application this will come in handy .... *potters off to tinker
hmmmmmm....10 minutes you say..very impressive...
the force is strong in this one...
grmph grmph... nothing more can I teach you then...grmph.
Beware the dark side young Jedi.
:)
@Ray - you don't need to buy your own Coopers if you come down under - there are plenty of people here that will buy you a beer or two..n
I'll hold you to that. :)