Note - the following applies to any multi file uploader - not just the one that ships with ColdFusion 9. Earlier today I read a great article by Dan Vega about some of the new UI controls in ColdFusion 9. One of them was the new multi file uploader. If you haven't had a chance to play with it, I highly encourage you take a quick look at the demos Dan uses in his article. Reading his article brought up some interesting things that I think people need to be aware of before making use of the control. I had a long talk with Dan about this, and unfortunately, the more we talked, the more we discovered that there are quite a few non-trivial things you need to concern yourself with before adding this feature to your site.
The primary, and most important, thing to keep in mind is that the multi file uploader performs a HTTP post by itself. Consider the following simple form.
<form action="form.cfm" method="post">
Name: <input type="text" name="name"><br/>
Email: <input type="text" name="email"><br/>
Attachments: <cffileupload url="uploadall.cfm" name="files" ><br/>
<input type="submit">
</form>
This example is different than what you see typically demonstrated with the control. However, it represents a more real world scenario. Users don't normally just upload files by themselves, but in context with other data as well. In this case my form has 3 "fields", a name, a email, and a attachments area. Our business logic needs to work with the data as whole. It will probably want to insert the two simple values into a table. It will probably then want to log the files attached to the submission in another table, where that tables links back to the original.
Right off the bat you can see the problem. The files post to uploadall.cfm. The form itself posts back to form.cfm. So on top of that - guess what - we have other problems as well. What if the user uploads files and never hits submit? What if the user picks files and never hits the upload button? As you can see - the more we approach a real world scenario, the less trivial this "simple" feature becomes. Let's talk about some possible solutions.
First - let's focus on how we can handle keeping the files together with the rest of the form submission. We have to accept that - at minimum - we may or may not have files waiting when the form is submitted. We need to also accept that we may have multiple people using the same form. Because of this, it may make sense to use a file location based on a value unique to the session. So for example:
<cfset session.storage = replace(createUUID(), "-","_","all")>
This creates a UUID based variable stored within the session scope. I called it "storage" which is a bit vague. If I had 2 or more forms on my site then I'd maybe want to use a prefix:
<cfset session.storage = "resumeform_" & replace(createUUID(), "-","_","all")>
We can then use this for our uploadall.cfm file.
<cfif not directoryExists("ram://#session.storage#")>
<cfdirectory action="create" directory="ram://#session.storage#">
</cfif>
<cffile action="upload" destination="ram://#session.storage#" nameconflict="makeunique"/>
Now that I have a place to store my files, I can look in that folder when I submit the rest of the form. However, this leads to more problems. (Does anything in life go from complex to simple??) First, what happens if I go to the form, upload some files, change my mind, come back and decide to start over? Most likely I want the form itself to remove all files that exist within my own storage directory. This means a simple page reload will also cause the storage folder to empty out. Secondly - what happens if I upload some files - go eat lunch - and then come back and complete the form? Well all of a sudden my storage system is empty. My form will think that no files were associated with the submission. If a form didn't require a login, then this could lead to confusion. What I'd probably suggest here is a warning if the form is submitted long after it was initially created. (You can use a hidden form field with the current time for this.) Then the user can decide if they need to bother to do the form again. Using this system also allows us to make use of onSessionEnd to empty out the folder when the session expires - again only if the user never bothers to submit the form. If they do we should process the folder immediately.
And unfortunately - we have another problem as well. When the multi file uploader sends data to the server, it does NOT pass along cookies. What does that mean? It means it uses another session. Luckily we can get around that by appending the session values to the URL:
<cffileupload url="uploadall.cfm?#urlEncodedFormat(session.urltoken)#" name="files" />
Note the use of urlEncodedFormat. This is required! Session.URLToken is URL safe already, but for some reason when I left this off, only the CFID portion was sent to the server.
Now that you have wrapped your brain around that issue - let's bring up another thing to consider. What if the user selects a few files, but never hits Upload? Unfortunately, there is no JavaScript API that let's you look at the list of files to see if anything is there. You are basically out of luck. The best I can recommend here is to put a nice message next to the control and by the submit button as well. Of course, users hate to read, but at some point they need to take some responsibility!
Finally - let me leave with two quick things I found while testing. First - if you use onComplete and forget to actually write the JavaScript function, the multi file control won't load at all. Firefox correctly reported an error, but with it "hidden" in the Error Console (boy that really bugs me - I wish Firefox would add something to to the status bar on errors) I completely missed this at first. Secondly - if you do not use the cffile action="upload" on your form process, then you will never get a response in the form control. I guess this is to be expected, but I thought I could simply ignore the file uploads while testing. Nope - when I had a blank file, the form control waited forever for a response. Again - not surprising.
Archived Comments
As a quick note - I plan on writing a full demo that demonstrates everything (up to the db storage, I'd fake that). This may be delayed though as I've got RIAUnleashed this week.
Would using hideUploadButton="true" somewhat simplifies the problem?
Nope. In that case, you would use the JS API to run the uploads. I didn't mean to imply there were NO JS API funcs for this UI, there are, just none that can get the status of the queue.
Thats a lot of "gotchas" for a feature that I really wish was easier to use... oh well.
Very interesting, I have not moved to CF9 yet (gonna be a while) but I was particularly looking forward to trying out the multifile upload. It seems to me that unlike many other things that CF makes easier, the multifile upload posses just as many issues as it solves.
BTW, I would be negligent if I did not suggest the firebug addon for firefox. It does in fact give you a message in the status bar telling you all about JS errors.
@Tyler: To be fair though - these gotchas apply to _any_ multifile uploader - not just the one shipping with CF9. To be honest, this is just one more thing that Ajax developers have to think about in general. Ajax isn't all just sunshine and puppies - it's a whole new way of thinking.
Yikes. Seems like some of the problems could be alleviated by having a multi-step submission process, which would allow you to use the cffileupload tag without a parent form wrapper.
Also, why doesn't CF doesn't send cookie data along with the uploads? Unless there's some good reason, I'd consider that a bug that needs fixing.
And I don't know if I agree with you about storing uploaded files in RAM. Too many simultaneous uploads and your virtual memory is toast.
@Russ: Even with a multistep process, you still need to coordinate keeping the files in one place and "gathering" them when the form is complete. You still have to worry about people not completing the form.
Definitely agree on it being a bug that cookies aren't sent.
RAM: Ram is cheap. ;) But certainly you could also save it to some other physical place.
Flash player (what CFFileUpload uses) never respects browser's session cookies.
@Ray I definitely like the idea of removing files at the end of the session, regardless of where they are stored. onSessionEnd() is rad.
RAM: Storing user uploads in it just seems like an unnecessary risk. If hackers didn't already have enough ways to screw your server, now they can eat up your RAM too.
Yeah but you get the exact same risk with the file system. You can file a file system just like you can a RAM virtual system. Of course the RAM one is smaller, but you still have to check the size. CF adds support for that with getVFSMetaData().
Comforting to learn that others are having as much difficulty getting this tag to work in a 'real life' app.
One problem I have identified relates to the cffile upload info array created. The CF9 documentation page for the tag states that 'After a file upload is completed, this tag creates an array of structures specified by the result parameter. Each structure in the array contains upload result information for one file'.
Unfortunately similar to someone who has added a comment to the info page, only the last element appears to be accessible, hence all the useful file details are lost, making subsequent manipulation and association a bit problematic
Paul - I'll have to test this myself. If you can't get data about all the files, then that is truly sucky and makes the feature 100% useless.
Ahah. Get this. I uploaded 5 files. The CFM that processes the upload is run 5 times. So you don't get 5 results at once - you get 5 separate results. There is no way to tell that they came in as a "unit". I don't necessarily think you _need_ to know that though. Point is - you DO get access to them all.
Ray
I am very impressed that you are on the case so early - it's 14:14 here in the UK!
I picked up that the upload runs almost like a cfloop hence the over writing of the cffile array perhaps.
As it runs like a loop I was hoping to pick out info from the cffile array or some form of counter variable as I am trying to rename the uploaded files at the same time. This covers the situation where a number of totally different filenames can be made into an identifiable set e.g.
front.jpg, side.jpg, top.jpg becomes elevation_01_01.jpg, elevation_01_02.jpg, elevation_01_03.jpg
At the moment I am working on the idea of throwing the uploaded files into a temp dir, interrogating this directory to get the file details, saving that in a structure within the session scope, processing said files (modifying my structure accordingly) then storing key items from this structure in the 'master' db record relating to this one process. Oh, then moving the modified files to my permanent image directory and deleting the original temp dir.
I was hoping that getting into the cffile array at time of upload as it loops through might have cut down the amount of code and hence been a bit more elegant.
Regards
Paul
As far as I know, it is not like cfloop. Rather, the UI control just does N requests. So the server side code, it's just getting one file at a time. I did a test with a regular form and 3 hard coded file upload fields. When I test with that, I do indeed get an array of 3 items. So file/action=uploadall can correctly handle N file uploads, but the _front_ end, when using the fancy uploader, is doing it simpler. Which kind of makes sense - you get more control if the uploads are one at a time. You can cancel a long running process for example.
I'm still planning on a _full_ demo of this feature, and I want to demo the simpler version I described above.
FYI, it's not so early here. ;) I was up at 5:45AM, my kids not much longer. :)
What I've had to do in most cases is provided the upload function once that data is created (so it's a two step process).
So if you have let's say "upload images" I create a gallery, then on each gallery item I provide an upload option... when you upload the images (for example) I tie them to the gallery, then, and only then, do i allow you to edit the details for the image. (There are other uploaders out there that alow you to have those fields with the image uploads so you can do it all in one, but the one in CF9 does not have that feature). A good example of this would be the one that FaceBook ad MySpace uses...)
Your first example assumes you want to do it in the traditional way (one action page to do it all)... but by leveraging the flash interface you are in fact doing two things (hence why this happens with ALL of them)... I agree it would be nice to have an interface that selects multiple-files, but gets submitted with the form, instead of in a two step process...
P
Just wanted to say great job with the blog, today is my first visit here and I’ve enjoyed reading your posts.
.... and all I ever wanted was a progress-bar for my file upload :) If this ability was some how built into the HTTP upload protocol, we wouldn't need all these special flash-based features.
Ben, there are a few kinda 'obvious' things I'm surprised aren't in HTML yet. For example, I think the IMG tag should have native support for a mouseover image. Something like:
<img src="foo.jpg" hover="foo_hot.jpg">
Sure it's trivial in JS, but mouseover images have been in use since my grandparents wrote HTML (*cough*) so it's a bit odd that simpler support for it hasn't been added.
Yeah, definitely.
@Ray having a "hover" attribute on the image tag is a terrible idea... I'm a firm believer that XHTML is for providing structure, and JS is for layering on behavior, and things like changing images on hover is certainly a behavior.
Besides, in what case would you want to hover an image to see a different one in which you would not also need JS to control placement, so a caption, or some other related thing?
We will have to agree to disagree here. I'm a firm believer in standards - but also a strong proponent of being practical. If a solution isn't "beautiful" in a pure comp sci sense but _practical_ for the folks who actually build web pages, then I would err on the side of practical (for the most part of course, not always!)
Is the requirement of urlencode() a bug to begin with? We were trying to upload to a Coldbox event and the URL is so malformed that it couldn't locate the existing session. e.g.
url="index.cfm?event=general.myUploadEvent&#urlEncodedFormat(session.urltoken)#"
ok, it seems like all the '&' in the url attribute needs to use urlencode(). Then the URL scope can parse all the key/value pairs, but unfortunately coldbox is still having problem with this urlencode() url somehow...
Hmmm. Now that I think of it, you shouldn't need to use it. It's supposed to be "URL Ready". Have you compared the value both 'as is' and wrapped in urlENcodedFormat to compare?
Hi Ray, for all of us on CF8 or Railo, do you have any suggestions for a multi-file uploader for us guys?
Thanx up front ;-)
Absolutely: Uploadify - http://uploadify.com. Used it for a project before and it worked great.
If I've never blogged on it here, let me know and I will try to.
Wow, Ray, this is amazing! Here I was, thinking I was all alone in this crisis! I anxiously await your full demo, as I am grappling with the very same issue -- not being able to submit form info along with my files. One attribute I thought would work is "onUploadComplete" -- thought I could stick a "document.form1.submit()" or something in there. Alas, when I use this attribute, the whole element disappears! What is "onUploadComplete" used for? How is it used?
What element disappears? onUploadComplete is used for what you think - running JS code when the file is done uploading.
Ray, just tested this, and it only seams to work if you are using J2EE session variables, since coldfusion sessions pass two variables CFID/CFTOKEN, it doesn't seam to work in that case.
What if you change
uploadall.cfm?#urlEncodedFormat(session.urltoken)#
to
urlSessionFormat('uploadall.cfm')
Ray, I don't think that would work because "If the client accepts cookies: does not append information", also I think it will still pass two parameters.
Hmm. So with J2EE turned off I get:
uploadall.cfm?CFID%3D10508%26CFTOKEN%3D99378812
I then got rid of urlencodedformat:
uploadall.cfm?CFID=10508&CFTOKEN=99378812
Then turned back on J2EE, kept the formatting off, and got:
uploadall.cfm?CFID=10508&CFTOKEN=99378812&jsessionid=8430d08da42160276820f3c7013e57281487
These last 2 look valid. So perhaps it is just a bug in my code with urlEncodedFormat?
Hmm, I guess I'm not sure why it seamed to work with J2EE session variables on then, I was thinking that the reason was that it only needs to pass jsessionid in the url, but it looks like it still passes cfid, cftoken.
So... are you agreeing I just need to remove the urlEncodedFormat()?
I'm having issues with cffileupload prompting me for credentials. My app is running on IIS and requires Windows authentication, but I have set the directory being uploaded to and the file that is handling the upload to allow anonymous access.
Nevertheless, I still am prompted, and if I cancel the prompt, get a 401 error. If I type my credentials, the uploader hangs and crashes the browser.
Anyone run into any permissions stuff with this tag? Pretty sure my code is ok, but here it is anyway.
<cffileupload url="dbaction.cfm?uuid=#url.uuid#&addfiles=1" bgcolor="767676" width="750" maxuploadsize="19" extensionfilter="*.pdf, *.xls, *.xlsx, *.doc, *.docx, *.rtf, *.ppt, *.pptx" />
<!---dbaction.cfm--->
<cffile action="uploadall" destination="c:\temp">
Double check with a network monitor, like ServiceCapture or Charles. It should be a HTTP POST call, and if the URL matches the folder in question that is unprotected, it shouldn't be prompting. You know this of course. Mainly I'm just asking you to double check your/our assumptions.
Ok, my bad. I feel stupid but will share anyway. It took looking at the ServiceCapture path to realize that I left off a directory in my "url" argument. (needed to be './handlers/dbaction.cfm' rather than 'dbaction.cfm')
Glad this happened though, did I uncover a bug? Apparently CF doesn't tell you when you try to upload to a non-existent page...Wouldn't it make more sense for a 404 to be thrown?
Where would it get 'thrown'? In the status area at the bottom of the UI control? Also note most users wouldn't know what a 404 is anyway.
I was thinking in the same place where the 401 error was displayed previously, in the progress bar on the file that failed.
Could be useful for the dev who is having brain trouble that day, unless I'm the only one who struggles with that from time to time :P
Sounds ok to me. File an ER. :)
Did you try specifying onerror="jsFunctionName"?
Well I think even if you _don't_ specify an error handler, by default the UI should indeed show that something had gone wrong.
something else I just noticed, you can only pass one parameter through to the action page. for example, I wanted one .cfm page to handle different situations, so url.addfile=1 fires off one section, url.appendfile=1 does another, etc.
I also needed to pass the UUID to my handler page so it knows what to name the folder...but it only reads the first parameter after the ? in my url
I may be getting my terms mixed up, so here's what I mean:
url="./handlers/file_manager.cfm?uuid=#url.uuid#&addfile=1" only sends the uuid, and "url="./handlers/file_manager.cfm?addfile=1&uuid=#url.uuid#" only sends addfile=1...
I guess I'll have to create separate pages for each action?
I don't quite get what you mean. It sounds like you want to do two things on upload, which is one issue. The second thing is that it sounds like you just want to pass 2 (or more) URL variables. If you cant, then that is a serious bug. Tell me - what happens if you use & to separate the values?
yeah that's what I mean, I am trying to pass 2 URL variables, separated by "&", and only the first is passed. the second and subsequent variables are lost.
So did you try &? If my blog messes that up, it is the ampersand followed by a, m, p and a semicolon.
yep, still nothing. I'm filing a bug report now
Ok. Dang that kind of sucks. Of course you can always make a struct of data and serializeJSON it (or encode it some way) and pass one arg.
My project works perfect in IE8, but Firefox and Chrome and maybe others won't remember me. In other words, I am uploading on a page that requires a login. I am logged in, but when I upload a file it forgets any login info on the upload.cfm page and shows the loginpage.cfm. IE8 somehow remembers it. I have tried the URLEncode(Session.URLToken), and using Fiddler it seems to be going across, but for some reason it doesn't do anything with it.
Any ideas of what I might check? (BTW, I also found out the hard way about it only sending 1 parameter across, even if you sent more)
Hmm. Can you add logging to onSessionStart and see if it is firing in FF/Chrome?
I decided to have a process folder where I call it. In reading others comments it seems that it doesn't pass the cookie variables across and so therefore my login doesn't work, because I am using cflogin with sessionmanagement and storing cookies. Anyway, I just pass an encrypted value across and decrypt and go on my way.
We're starting to run into something similiar with ColdFusion 9 vs ColdFusion 8. Basically, a couple years ago we built our own Flash multi-file uploader, and ran into the "Session not carried across in Firefox" issue, so I simply appended the current Session.cfid and Session.cftoken to the URL called by the Flash SWF, and everything worked fine in CF8.
Then came CF9, and now it doesn't work; we originally thought that a Firefox update was behind the issue, but today while testing and looking in my IIS logs, I see that the POST requests do indeed reach IIS, and the URL carries the correct session CFID and CFTOKEN vars. But when logging to the administrator log files, the actual Session.CFID and Session.CFTOKEN variables end up with different values... basically, Flash's own session.
Perplexed as to why the URL variables were now being ignored, whereas they weren't before, I found this blurb part way down the Adobe CF9 Client State Management docs (http://tinyurl.com/2c6jyfy):
Note: The behavior is as follows when CFID and CFTOKEN are provided in the URL: If session exists, the CFID and CFTOKEN from the URL are ignored. If the session does not exist, CFID and CFTOKEN from the URL are used to validate the session and the session is used if it is valid. If the session is not valid, a new session is created. CFID and CFTOKEN are regenerated.
So, it would seem that CF9 changed it's behavior from CF8, where the URL variables overrode any other session variables. That seems odd to me, but it's the best I could come up with today... let me know if you know any differently, Ray!
Question - does it work if you use session.urltoken instead? That's the proper way (afaik), instead of manually doing cfid/cftoken.
I'll have to try that, Ray. However, it definitely looks like a change from CF9 to CF8; I ran some tests today by manipulating the URL variables to change sessions between several browsers, using CF8 and CF9 servers, and it all worked in CF8, but was ignored in CF9. (I blogged about my results at http://blog.pelcosolutions.com, a bit long winded, skip to the test stuff, and see if your results are the same!).
I'll let you know if the Session.urltoken stuff makes a difference, but I'm guessing that this change just makes CF9 more secure, and it won't? Not necessarily a bad thing, just a change! ;)
I'm not sure it is more secure per se. Remember that since CF6, we've had j2ee sessions as an option, and that uses different variables. session.urltoken works with both and is 'safe' to use.
cffileupload doesn't seem to be working on chrome. it works fine in ie/ff. is there a work around? i keep getting status code: 302
I -only- use Chrome and have not seen any issues. Got an example online that I can hit?
I get a status code : 302 error if on my upload.cfm page I have <cflocation url="formSelector.cfm?id=#session.userObj.trackerID#" addtoken="no" />
I need to change pages after the files are uploaded
The multifile uploader uses a background network process to upload the files. You can't cflocate that because it isn't your main browser's request per se. It isn't an Ajax XHR request, but you can think of it the same. So if your need is to go someplace after done uploading, use the onuploadcomplete attribute of cffileupload to specify a JS function to run.
So I had this working on 9.0. I upgraded to 9.0.1 on Monday and both parts in my code where I use this breaks. The problem is that the variables aren't getting passed to the post upload code. I tried both with and without the url encoding. No luck. Anyone know of a bug or workaround? I'm using coldbox.
Sample:
<cfset request.theURLVars = urlEncodedFormat("teamid=#event.getvalue('teamid',0)#&linkid=#event.getvalue('linkid',0)#") />
<cffileupload progressbar="true" stoponerror="true" maxfileselect="20" title="Photo Gallery Image Upload"
extensionfilter = "jpg,png,gif" deletebuttonlabel="Remove from upload list"
clearbuttonlabel="Clear upload list" url="/upload/uploadPhotoGalleryProcess.cfm?#request.theURLVars#"/>
Have you tried a network monitor like Charles or Service Capture to see the request being made?
I resolved the issue. I found that I was able to now pass more than one url variable and didn't need the url encoding.
Great stuff as always, Ray. About the link to Dan's article in your entry, though, it fails. It may not be obvious to some, but if you just take off the _02 at the end of the filename, it works: http://www.adobe.com/devnet...
Ray, if you want to just edit the entry and delete this comment, that's fine.
I had that same link a bit earlier so I just removed the second link. Thanks Charlie.
Why i got error status code:302 in firefox and chrome but in internet explorer was working.
<cffileupload url="uploaddocs.cfm" progressbar="true" name="myupload" addButtonLabel = "Add File" clearButtonlabel = "Clear it" hideUploadButton = "false" width=600 height=400 title = "Add New Document" BGCOLOR="##FFFFFF" MAXFILESELECT=10 UPLOADBUTTONLABEL="Upload Files" stopOnError=false/>
uploaddocs.cfm
<cfscript>
thisPath = ExpandPath("*.*");
thisDirectory = GetDirectoryFromPath(thisPath);
FileDir = thisDirectory & "uploads";
DocRoot = request.root;
RootDir = GetDirectoryFromPath(DocRoot);
</cfscript>
<cfif not DirectoryExists(FileDir)>
<cfdirectory action="create" directory="#FileDir#" >
</cfif>
<cffile action="upload" filefield="fileData" destination = "#FileDir#" nameconflict="makeunique" mode="777">
I was searching on the net and got here
url="someurl.cfm?#urlEncodedFormat(session.urltoken)#"
then i just added it and it worked.
Thank's
Greetings!
I've recently encountered this blog and it's assets for the CFFileupload. Does anyone have any suggestions for multiple file download? i.e. CFFiledownload?
Many Thanks!
Well, it depends on what you mean. You can't send N files at once to a user. You can send one at a time via click. You can send N files in a zip file too. I suppose a Flash-based downloader would possibly allow you to send N files to the user at once, but that would be a pretty scary prospect imo.
Thank you for your response!