Tom asked,
Ray, one topic that I'd love to hear about from the audience on your site is the techniques that are used to send files live on a production environment. Do people use a program like xcopy or Robocopy, or do they just push them manually with FTP? What kinds of testing and versioning are commonly used before a file is sent live?
To be honest I suck at this. Well, I don't know if "suck" is the right term, but I don't do anything fancy. I just use FTP. Typically, once a site has gone live, my only changes are new features and bug reports. (Sorry, "Issue Management.") Since this typically involves one or two files, I just FTP them up to live. This has only been a problem once - with a client that had two boxes, and they had replication set up so even if I did forget to move a file, it was fixed within the hour.
You mentioned versioning as well, and yes, I do use versioning (mostly Subversion now), but it isn't tied yet to my deployment process. (If that makes sense.)
So this is probably not the best answer - but as Tom said, he was looking for a range of answers from my readers. (As a side note - how do folks feel about "discussion" topics like this? I almost didn't blog this as I didn't have an 'authoritative' type answer, but he did request feedback from my 'audience' so I figured it was ok. This blog is certainly more than just me, and I love that, and I think my readers do as well. Anyway, if folks have a problem with entries like this, please ping me via email.)
Archived Comments
I used Beyond Compare, from http://www.scootersoftware.com.
Very user friendly, and has automated processes.
I make a copy of "everything that's changed since yesterday".
The more you read the manual, the more things you can do with it.
I use Subversion as well - at some point I want to look at ANT as I think it's possible to script this process which would make things really easy... I just haven't had the time :(
For large enterprise deployment, you probably want to look into ANT or MyEclipse Sync-On-Demand.
For single server setups with single directory configuration, using standard remote server in Dreamweaver will work fine in most cases. I often work with a dev directory and once the new features are working fine, I usually diff the changes with something like Visual Slick Edit and only copy what I need to.
And there's always the CF editor a lot of us cut our teeth on, CFStudio/Homesite+, which still supports scriptable deployment and has a wizard that automatically builds the scripts for you.
I've always wondered who and how uses the "Packaging & Deployment > ColdFusion Archives" in CF admin? Can .car file deployment be done with ant tasks or is it manually via the admin interface only? How does it work with clustered CF servers?
Anyone with experience in CF .car/.ear/.war deployment care to comment?
I've used CARs, but only to back up settings before a new CF install.
Right now I have a script that exports a tag from CVS/SVN and pushes to prod server(s) (they are on the same network).
I think the biggest problem that Tom is having is overthinking the situation. I'm a one man shop. I use Subversion and FTP my changes to the server. Probably 99.9999% of you out there do the same. Am I going to use ANT, write custom script or automated programs to do this for me? No, of course not, Why? Well because the amount of time it would take me to learn the scripting language, write and test the scripts, would take away from my development time. Now if you're a big time shop, with a lot of developers, working on different branches and pataches, then you want to invest the time to automate the deployment process. So Tom, stick with what you got and don't waste your time trying to "fix" or improve a process unless you can get some ROI out of it.
You mean things actually get out of development and get deployed?!
----
As a one man shop, I primarily use FTP. Whenever possible, I use secure ftp. I like <a href="http://www.winscp.com/">WinScp (see winscp.com)</a>.
My sop used to be to encrypt (apparently cf7 can bytecode) the source until final payment was received. I became lazy and simply trust my clients now. Not protecting source in someway has cost me 2 or 3 times.
It's surprisingly easy to add an "Automatic Update"-type of feature to your deployed apps, especially now that CF has built-in message hashing (checksum) functions:
1. Make a nightly process on the deployment server (updates.mycompany.com) that does:
a. MD5/SHA your production source tree. (md5deep is great for this, but it can also be done in CF)
b. Add a database entry for each file that has changed.
c. Optionally, add a text field that holds a "diff -U" for ease of debugging.
2. Make a script on the deployed app that does:
a. Call (via RPC or RSS or whatever) the deployment server to find updated files.
b. Fetch the updates (via FTP or HTTP)
c. Check the updates for integrity (MD5/SHA)
d. Log the updates
e. Backup the to-be-replaced files
f. Optionally, install the updates or email an admin to visit a "finish my updates" page
3. Schedule said script to run nightly/weekly/whatever.
I rigged something like this up in under a day at one of the companies I used to work for. I think it was a grand total of 4 CF files:
- Nightly checker (deployment)
- Update checker (client)
- Script server (deployment)
- Finish-Your-Updates page (client)
It was unbelievably useful for keeping our multitude of clients in sync. Since you are only pushing files to production after you've tested them thoroughly (one would hope), you'll have very few breakages. It also makes you keep your code in one trunk instead of forking it off for every client.
Yes, there are security holes here and there, but nothing that can't be plugged with a bit of thought. Code signing, for example, was a bit heavier than we needed -- if someone was going to use DNS poisoning to get their files on our client's server, our client had bigger problems they needed to sort out. However, client authentication and IP verification on the deployment server were very easy to work in.
Every one of my projects has a build.xml file. I can then use ANT (built into eclipse) to do the deploy. I like it because it guaranties that my deploy is the same every time. No worries about forgetting to copy a file or two. I know very little about ANT, so I’m sure there are many more things I could do, and probably better ways. Here is an example of one of my build file. This file checks out the latest version of the code from subversion, creates a tag of this version in the repository, and then copies it to the production server.
<project
name="projectName" default="build" basedir=".">
<target name="init" description="Set Props">
<!-- Define build vars -->
<property
name="project.name" value="projectName" />
<property name="project.build.root" value="c:\Build"
/>
<!-- Define destination targets -->
<property
name="dest" location="\\server.com\c$" />
<property name="dest.www"
location="${dest}\Inetpub\wwwroot\${project.name}" />
<property name="dest.inc"
location="${dest}\Inetpub\include\${project.name}_inc" />
<property name="dest.com" location="${dest}\Inetpub\com"
/>
<!-- Create the time stamp. -->
<tstamp>
<format property="buildTime"
pattern="yyyyMMdd_hhmmss_aa" />
</tstamp>
<!-- Define svn data. -->
<property name="svn.rooturl"
value="svn://server.xyz.com/" />
<property
name="svn.username" value="yourUserName" />
<property name="svn.password" value="yourPassword" />
<property name="svn.projecturl"
value="${svn.rooturl}${project.name}/trunk/" />
<property
name="svn.comurl" value="${svn.rooturl}com/" />
<property name="svn.tagurl"
value="${svn.rooturl}${project.name}/tags/${buildTime}/" />
<property name="svn.revision" value="HEAD" />
<!-- create build dirs -->
<mkdir dir="${dest.www}"
/>
<mkdir dir="${dest.inc}" />
<mkdir
dir="${dest.com}" />
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="cleanBuildDir" description="Cleans the build
directory">
<delete dir="${project.build.root}" />
</target>
<!--+++++++++++++++++++++++++++-->
<target name="checkoutBuild" description="Pulls code from
Subversion into the build directory">
<exec
executable="svn">
<arg line="export
${svn.projecturl} ${project.build.root} -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
<exec executable="svn">
<arg line="export
${svn.comurl} ${project.build.root}/com -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
<exec executable="svn">
<arg line="copy
${svn.projecturl} ${svn.tagurl} -m 'Copy To Tag' -r ${svn.revision} --username
${svn.username} --password ${svn.password}" />
</exec>
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="copyWWW" description="copies www files">
<copy todir="${dest.www}_bu" overwrite="true">
<fileset dir="${dest.www}" />
</copy>
<copy todir="${dest.www}" overwrite="true">
<fileset dir="${project.build.root}\wwwroot">
<exclude name="build.xml" />
<exclude
name=".project" />
<exclude name=".psd"
/>
<exclude name=".doc" />
<exclude name="test.cfm" />
<exclude
name="**/sql**" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target
name="copyInc" description="copies inc files">
<copy todir="${dest.inc}_bu" overwrite="true">
<fileset dir="${dest.inc}" />
</copy>
<copy todir="${dest.inc}" overwrite="true">
<fileset dir="${project.build.root}\inc">
<exclude name=".project" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target name="copyCom"
description="copies com files">
<copy
todir="${dest.com}_bu" overwrite="true">
<fileset dir="${dest.com}" />
</copy>
<copy todir="${dest.com}" overwrite="true">
<fileset dir="${project.build.root}\com">
<exclude name=".project" />
</fileset>
</copy>
</target>
<!--+++++++++++++++++++++++++++-->
<target name="build"
depends="init,cleanBuildDir,checkoutBuild,copyWWW,copyInc,copyCom"
description="Builds Project" />
</project>
Tony-- thanks for your thoughts. I think that you're correct that for most situations the use of FTP and Subversion will be adequate. But I would caution you not to mistake research and dialog with "overthinking". For instance, I used to think that source control was too complex to learn and administer. So I didn't realize until just last year, when I first set it up, how easy and powerful Subversion is. Then I got to wondering if there were other deployment techniques out there that would be well worth my time.
I could be misinterpreting your comment, but I wouldn't agree that people shouldn't investigate other deployment strategies just because they're already using FTP and source control. There could be some other application or strategy that neither you nor I know about that would be well worth the time to explore. Now, making your deployment overly complex and administration-intensive-- that's overthinking. But you'll never know unless you investigate a little.
Using Subversion (or any source control) is one of those things I wish I had done a LOOONG time ago! Sure it takes some time to initially setup a project but that is easily made up once I screw something up and can easily roll back my changes.
An automated deploy script would be just as beneficial - how many times have you FTP'd files and forgot something? Once that is scripted you would no longer have to worry about that.
Tim - thanks for the example script - that's going to get me motivated to RTFM for ANT.
ANT is very slick. It can repeatably do all the steps in your deployment, such as: pull specific version of source repository, remove test files or other supporting files, alter XML or config values to production settings, zip the code, copy it to the target server, zip and backup existing production code, enable a "site maintenance" type default page, delete the existing production code, unzip the new production code, and execute any reload URLs to refresh applicaiton scopes, etc. It takes a while to tweak the ANT build.xml to do this but once you have it working for your set of tasks, you can do all of this instantly with the push of a button.
Two quick questions:
1) can ANT use FTP or SSH for moving files around?
2) I primarily using CFEclipse with the Subclipse plugin to code and manage SVN - is there an ANT plugin?
Yes, ANT will perform tasks using FTP and SSH. Full details in the ANT manual online at:
http://ant.apache.org/manual/
There is no need for a plugin becuase ANT is built into Eclipse (though you can also download and run it separately if you wish).
Jim,
Depending on the version of eclipse that you have installed (I always use the one thats about 100MB), ANT is all ready built in. I guess the easiest way to check is to create a build.xml file. If you see an ant icon on that file (literally a picture of an insect).
When you right click on this file, you should see a "Run AS -> ANT Build" option. Select that and watch the magic happen.
Do you create the build.xml file in your ColdFusion project root dir? I checked my Eclipse plugin list and I see 'Ant 1.6.5' so it looks like that is OK. I can already see the afternoon is shot until I get this working!
We have been using CVS for over a year now, I had to force it on the team :) We don't need nightly builds, releasing code twice a week is fine. We use Remedy internally for issue tracking and resolution. We simply tag the CVS module with the Remedy issue number, the d00d in charge of production release just updates the module direct from CVS based on this tag...whammo! With this we can just as quickly and effortlessly roll a release back too.
works great, but I am thinking of using Ant eventually for this...Ant is just uber slick.
...and we have a site wide license of BeyondCompare, as said above, this thing is slick and powerful.
DK
Jim,
I put the build.xml file directly in the webroot. Notice this in my script : <exclude name="build.xml" /> . This command prevent the file from being copied to production.
Any thoughts on checking out code directly into the production environment?
We only "deploy" to one spot: The production web server. Deploying to some staging area and copying up seems redundant to me.
----
Ray, please keep posting discussion topics. You don't have to have the answers if you can moderate a discussion. I learn most from blogs that invite me to participate.
Because we'll do deployments at werid times during the day and frequently, we've turn our production servers into a Subversion working copy. This give us the ability to 'swtich' to newer revisions as needed. It also allows us to do an immediate rollback if the newer revision is causing problems. The account used to roll out changes only has 'read' rights to the Subversion repository. This keeps accidental or malicous commit to the repository from happening via the production web servers.
To do this though, you must fully use the trunk, tags, and branches methodology for Subversion. Since trunk is always our best working version, we deploy from the trunk. Others who do more 'scheduled' release might 'switch' using a branch or tag path.
I've been using Subversion for about a month and it has already helped me to avoid disaster on more than one occasion. Being that I am still a fairly novice Subversion user and new to version control systems altogether, this question may have a very simple answer...
I have a large web site with a complex directory structure under version control and the site requires frequent updates and deployments via FTP. Using a shell extension like TortoiseSVN, it isn't a pain to visually locate one or two files that have changed and upload them to the production server but this process becomes messy and error-prone if many files in many different directories have been changed.
How has someone else dealt with this successfully? Just do a bulk upload by selecting complete directories and hoping that your FTP program of choice can filter out all of the hidden ".svn" specific files? Is it possible to export unversioned from the repository only the files that have changed since a last revision while also preserving the directory structure if necessary?
I may be making this way more difficult than it needs to be but am interested to hear what others have done...
Thanks a ton,
Tom
I just code directly on the production server...just kidding.
I recently discovered Ant (I had heard of it before, but never really knew what it was or what it did) and hope to start using it in my deployment process.
Don't kid, Scott. Don't forget that the 5th largest site in the world does that. (Ahem)
You know, I've been giving this topic a lot of thought lately myself. We have a couple web servers we've opened up for hosting small departmental web sites. Our clients range from amatuer CF developers to folks that write a page in Word and save as HTML.
In years past, these clients developed on a Production server, but when the server's lease expired, we changed it out for two servers - one development and one production.
Now we're working through the process of moving a users coded from development to production. At the moment, we're doing it the old fashioned way -- they call me, and I move it for them, but that's getting really old, really fast.
We don't want to grant the users access to the prod box because we don't want to tempt them to change code there directly. Often times, these sites update several times a day as new Excel or Word reports are published or vacation calendars are updated.
For some users, we've set aside space on a file server for such content and used IIS virtual directories to include the info simultaneously in both the dev and production without having to move anything around.
Both servers are Windows 2003, running CF7 and IIS 6. Our backend is SQL Server 2000 and we're not using any sort of version control. Just curious to know if anyone has any suggestions on how best to accomplish the goal of moving data from one environment to the other, automatically.
Thanks,
Bob Weston
bobwestonATsbcglobalDOTnet
test
Geez - has it been a year?
I wanted to follow up on this post and say I've been using ANT now for several months to move my files from development into production and it's been working great.
I have lots of Ant info on my wiki if anyone is interested:
http://www.thecrumb.com/wik...
HI Ray,
I am getting below erorr while deploying CF6 CAR file into CF 8
We are planning to migrate from Coldfusion Mx 6.1 to Coldfusion 8 While we deploying the CAR file created from MX 6.1 into Coldfusion , None of the SQL datasource is working . Log file containts the below erorr message Error occurred while updating datasources:tedy" "Error","jrpp-10","08/20/08","13:49:36",,"An error occurred while trying to encrypt or decrypt your input string: com.rsa.jsafe.crypto.dr: Could not perform unpadding: invalid pad byte.. " After adding this in JVM config below error occurs -Dcoldfusion.disablejsafe=true Complete JVM Args java.args=-server -Xmx512m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.classPath={application.home}/../lib/updates,{application.home}/../lib,{application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,{application.home}/../wwwroot/WEB-INF/cfform/jars -Dcoldfusion.libPath={application.home}/../lib -Dcoldfusion.disablejsafe=true An error occurred while trying to encrypt or decrypt your input string: Given final block not properly padded. "
GSCopy Pro v6.0 (RoboCopy Alternative) with Open File Agent
GSCopyPro is a single command-line tool (CLI) that can copy, replicate and move files from one folder to another. This folder can be on the same machine/ server or another server elsewhere. What makes GSCopyPro stand out from other competitors is the fact it works on 32-bit as well as 64-bit systems and has no restrictions. It can easily be scheduled to run as a scheduled task and fully automated. GSCopyPro also comes with an open file agent which can copy files that are locked/ opened by other processes. This feature is supported in all windows vSCersions from widows XP/ 2003 and later.
Go To:>> http://www.gurusquad.com/GO...