Tim asks:
We have a clustered environment for testing and production; each cluster has two JRun4 servers and each of those has many instances of CFMX6.1. A given CF application will therefore live on both the JRun4 boxes. If a user were to change an application-scoped variable (while, obviously, being served by one box), how can we programmatically propagate that change to the other JRun4 server?
This is a pretty complex issue, and I feel like a good one to talk about since I think others will have good ideas as well. (That's a nice way of sayimg, I'll try to answer, but let's see what other folks say as well. :)
The first thing I would ask is - is your Application data coming from some central source, like a database? If your data is being stored in a backend system that is loaded when your application starts up, then you could build a web service that would force a refresh on the 'other' server. When the change is made on server A, your code would call out to server B and ask it to refresh itself. The problem with this solution is that it is a bit manual. You would want a solution that: contains a list of boxes - knows which machine is currently running (should be doable via CGI) - and then pings all the others.
What if your data is not stored in a DB? Maybe you have something like, currentBackgroundColor, and want to change it and then store it in RAM, not caring about the expiration of the variable. You could use a simular approach to what I described above, except this time, the message isn't to refresh the cache, but to pass a new value along.
Along with using web services, you could also store messages in the database. For example, when server A changes application.bgcolor, a message could be stored in a database table. The message would contain the information necessary to update the data, and a timestamp. Machine B could check for new messages in the table based on time. In this setup, you have a bit of overhead since you need to check the table every request, while in the first approach, you simply listen in for messages.
Hopefully this gives you some food for thought, and I know I have some smart readers out there who will come up with some better solutions.
Archived Comments
Session sharing is automatic between the Jrun servers, right? Perhaps a non-database solution, then, is to let the tail wag the dog. When the user changes the background color, the change should be made in the session scope, not the application scope. Have the session scope contain a "dirty" flag, along with a container of some sort (a transfer-object, I suppose) holding details of the requested change.
When the application sees the "dirty flag," it knows there's a pending message requesting an application variable change. The container holding the request is parsed, compared to the current application variable, and (if needed) the application variable is updated accordingly. So we end up with a sort of implicit invocation message-passing system whereby the application var sets itself based on a message planted in the session.
Since the session is hopefully replicated automatically to the other server via Jrun, the other server will also pick up the change and adjust its application scope accordingly. If you want to get fancy, you could add serial numbers or UUID version stamping to manage concurrency, prevent feedback loops, etc.
Just a thought.
-Ken
Ken, a problem with this. If I set a flag in the session scope, then yes it may exist on server B, but only in my scope. If I don't hit server B, it won't be reflected. I'd have to manually go to B, at which point the code there would see the dirty flag in my session scope.
Hm... that's a good point.
Here's a question: even if you don't end up on server B, aren't the servers still exchanging session data? If so, that means the "change packet" does exist on server B shortly after it's added on server A -- albeit without the animating force of your actually hitting server B. So perhaps it's a matter of building a sort of "listening service" that directly inspects session scopes on the server, looking for that dirty-flag message...
-Ken
back in the pre-JRun days we built an app that was designed to run on a dumb, round-robin DNS cluster. the machines knew nothing about one another.
when each CF instance was started up, code in the server's application file would register the server's existence with the central DB, logging a unique identifier like IP/instance name/hostname along with last updated time. periodically each server could be set to 'check in' with the 'registered servers' DB to let the DB know that the server was still running OK.
a couple of alternatives were considered to kick off the update: either a flag was set in the table to tell the other 'live' servers to update themselves (recurrent DB overhead as each server checked in) or something like a CFHTTP/Web Services call was looped across the IPs of all other servers in the 'registered servers' table to tell each server about the new data.
it's pretty much an amalgamation of the two techniques you'd already highlighted. it worked well anyhow - the app's still alive 4 years on.
I'll chime in with my two cents (I expect change).
I've done this numerous times before and the only solution I've found is to do it manually. That is to write and retrieve the settings from a database or other shared resource. After retrieving the information you can then write the information to scope variables on each server.
This to me is the safest way for all the servers in the cluster to get the same information.
This is a problem that we are facing as well. The particular scenario we face is that we have a cluster of three or four servers, (running a big CMS) in 'normal' (not J2EE) mode, with a load balancer in front.
Now the thing is, our load has reached the point where it's starting to get impractical to get impractical to render each page fresh from the database.
We have a caching system for html fragments, but at the moment, we are using a shared folder between the machines, and the performance gain of that is dubious (Filesystem access vs db access).
We need to try and move the cache to memory, but then, update propagation becomes the problem:
Problem one: How do you propagate an update? Fire off a TO with the new information? Or just a note to say invalidate the updated entry, and refresh it itself?
Problem two: What topology would you use to coordinate the updates? Peer to Peer? Or have a master cache server, with the other servers using the master cache for data access instead of querying the database directly? Or should the master cache be just a controller, relaying messages, and the servers should do their own data access?
Problem three: My concern with using web services is the extra overhead of throwing all that XML around. I'm thinking it might be quicker to send things around using JMS, but how does one go about hooking that into coldfusion?
Or am I overengineering the whole thing, and there's a simpler way to do this?
Apologies if I'm going off at a tangent, but this is a problem that I've been thinking about for a while, and you've just provided a way for me to let it out :D
We do this today with complex structures of arrays by using WDDX and a timer.
Every minute the first server (cf5) serialize the application structure into a WDDX-package, and the second server (cf7) collects it by a cfhttp-call. Of course, this causes the data only to be updated every minute on the second server, but it's not a big problem.
It works surprisingly well, and we have 8 000 000 page impressions weekly on these two servers. No problems to date.