Please forgive me but the following will be a bit long and lot 'rambly' so take this as you cue to stop reading now. I just took the ColdFusion 8 Certification exam and I have some thoughts I want to share. I also have something I want to share about the Dreamweaver certification as well. Before I get into this though, let me be clear. This blog entry is not meant to be on the relative merits of certifications in general. That's a whole other blog post, and frankly, you won't get much agreement in the community. My personal take is that I don't think a certification stands for much. It does show you can memorize well (which I tend to stink at), but it doesn't show how well you attack problems. I wouldn't ask a developer to rattle off the arguments to CFFILE. Instead, I'd tell the developer: "Imagine we have a cfpoo tag. You know it does X but can't remember the syntax. What do you do?" For me the answer is that I've got the CFML Reference PDF in my Finder so I can launch it whenever I need it - and trust me - I use this PDF all day long. I think only recently have I truly memorized the order of the Find() args, and Find() is - what - 8 years old? So yes, I suck at memorization.
On the flip side - if someone asks me if they should get certified, I generally recommend it if they are looking for a job. While I don't put much stock in certifications, other people do. And if a simple, 1 hour test for a hundred bucks or so (I have no idea) can help you land the job, then it's worth the time. If it puts you above some else's resume in a pile, then it's worth it. But if you've got a job and have no plans on leaving it, I wouldn't bother.
So I ask people - please - let's not discuss if certifications make sense. I really want to focus this specifically on the ColdFusion and Dreamweaver certifications and Adobe. Wow, that's a lot of writing for just a disclaimer, but I did warn you.
Let me give a bit of history on my relationship with the ColdFusion certification. I had experience working on the cert back in the CFMX time frame I believe. I wrote a few questions. I forget which company handled the process, all I remember is that they had a horrible web site. I've taken the certification a few times. I believe I took it for 5 and MX7 only though. (For the MX test I believe I got an automatic certification.)
Last year I was approached by a company (I'm not going to name them so don't ask) to help work on the CF8 certification. I wrote a few sample questions but they were not acceptable to the company. The reasons for this are not really relevant I think, and I place the blame more on me then them. I think I write ok - I've got about 12 books under my belt and way too many blog entries. But I find that when I have to write in a particular style or to a form - I find it extremely difficult. In this case, the philosophy behind some of the rules the company had were not something I agreed with, and we both agreed to part ways. I think they used maybe 4 of my questions or so.
Fast forward a few months and the company approached me again, this time with an offer to review. I love to review. I mean, I really, really love to do technical reviews. Its an open license to go ape-crap on a document and live out my anal-retentive fantasies to the max. I recommended two other folks (again, not naming names, will let them announce themselves if they want) and we all did an independent review.
What we found was pretty scary. Numerous questions were poorly written, wrong, or had multiple right answers. In a long phone session (late at night thank you very much) we hashed out the problems we found. Personally I was concerned that the company would not take all our criticisms seriously. I contacted Adobe about this as well. I felt that the certification needed some drastic oversight and additional review before it was made public.
Today I had the opportunity to take the certification. First - my score: 92%. I was a bit surprised by this. I thought I had done worse actually. I was surprised to find that - as far as I could tell - none of the recommendations myself and the other two testers had found were made. I probably commented on 15-20 of the 64 questions I took. Some comments were very simple - typos. Others were more serious.
As an example, numerous questions would say, "You do X and something goes wrong." Well shoot, what went wrong? The actual error has a big impact on what the problem is. If a coworker came to me and said that I'd give them a puzzled look and ask them if they knew what the actual error was. A related version of this were questions that said "You did X and it didn't work." Well, "not working" can mean so many things. Do you mean that an exception was thrown or that an unexpected result was returned?
Along with questions that were overly vague, I also saw questions with multiple right answers. In most cases I was able to guess, but I know that some of my missed questions on the test were due to this unnecessary confusion. To be fair - it's hard to write test questions. Shoot, I gave up! You can't use answers that are so crazy that the question becomes too simple. At the same time, if a question can lead multiple people to think there are multiple right answers, then a test taker should not have to spend time figuring out what the tester maker wants. It's not a "Can you read my mind" test but a "Do I know ColdFusion" test.
On a picky note - I was a bit surprised to see Application.cfm referenced. The test does make use of Application.cfc, and I do think developers should know about the existence of Application.cfm, but all focus should be to App.cfc I think. (This point I think I may get some push back on from my readers, and on reflection, I'm not even sure how strongly I feel about it, but I thought I'd put it out there.)
I was also a bit worried that the test wouldn't have any ColdFusion 8. I think I got to about a third of the way through before I saw something specific to ColdFusion 8, but after there was a good mix of CF8 stuff. To be fair, ColdFusion has a large number of functions and features and there is no way every question could refer to just version 8.
So let me summarize this before moving on to Dreamweaver. All in all, I think the test needs a lot of work. I do not think it should be taken in it's current state. It is not quite as bad as I remember it from the review, but I cannot recommend someone pay for this test unless they have to for a good job. I can stand on my high horse all day and say what I want, but don't be stupid. If you need the certification to pass, then take it. But outside of that, I'm suggesting folks avoid the test. This is my personal opinion obviously and I welcome opinions to the contrary, and as the subject says, Adobe, please, tell me I'm wrong.
So - the above is nothing terribly urgent. Not the end of the world for sure. ColdFusion isn't going to die because of it. However the following does concern me and frankly, disappoints me.
On another listserv I belong to, there has been a lot of discussion about the Dreamweaver certification. As I don't use Dreamweaver, I only paid it half-mind, but I wasn't surprised to see people complaining about it. I would assume the same people responsible for the CF cert also handle the DW cert, but I could be wrong. What did get my attention though was the concern amongst instructors. Apparently they have to pass the certification in order to keep teaching.
For them - a poorly written certification isn't an academic concern but a matter of their livelihood. Recently Dee Sadler shared some "good" news. Adobe had recognized that the certification was flawed, and since they didn't have time to address it, they were letting instructors off the hook. This is great. But then something caught my eye. Adobe went on to say that if an instructor wanted to be product certified, they still needed to take the test.
So.... consider that statement.
Adobe recognizes the test is flawed (good) and recognizes they don't have time to fix it (understandable) and will allow instructors to skip it (great), but they will continue to allow people to pay for and take the certification that they themselves admit is flawed and not appropriate for their instructors.
This boggled my mind. You can't say a test is flawed so much that it can (perhaps should) be avoided and still allow the test to be taken. These tests cost money. Maybe not a lot, but they do cost money. To keep these tests on the market is wrong. I understand that Adobe is a very large company and large companies move slowly, but I'm surprised that no more action has been taken. (That I know of, but that's part of the reason I'm blogging all of this - to help spur action.)
So - there we have it. I welcome one and all to come in and argue with these points. My goal here isn't to attack Adobe. That would be dumb considering how much I love their products. But I do feel that something is seriously flawed and that they need to step up and fix the issue.
Archived Comments
Thanks Ray for the update on the certification on Coldfusion. I was thinking of taking it, but I think I will hold off. You make a good point about if you don't plan on changing jobs, why bother.
Thanks for the heads-up on the ColdFusion exam, Ray.
My organization is just starting a professional development initiative where they'll pay for us to get certified in our respective technologies, so I was planning to take advantage of it and take the test, but doesn't sound like it would be worth it.
Thanks for your insight Ray. I have been planning to take the CF8 exam but if it's a low standard poorly written exam which doesn't hold up in the CF community itself, then it's probably not really worth the time and money.
I took a ColdFusion competency test for a job from a qualified testing agency a while back to determine whether I had the skills of a "certified CF developer " ...
After the test results came back, the hiring authority informed me that I tested out as "advanced" :)
Thing is ... I know that I am most certainly NOT an advanced ColdFusion developer ... maybe advanced beginner ... but no where near advanced.
Moral of the story?
Being able to take a tests well, does not translate in to skills in practice ~ vis a vis.
Food for thought.
Ray,
One of the questions I ask when interviewing is whether or not the person is certified. If they are, I ask them about the test, what they thought, etc. If they didn't, I ask them if they plan to (why or why not)... Their ideas on certification, motivations, and plans say a lot. So I don't necessary care if they are certified, but I use the certification process to start a discussion, and it leads to some very interesting answers. So I find it useful, but in my own way.
On the Dreamweaver instructor issue: currently certified instructors can take a "recertification" exam that is shorter and that we can take online from home. The test is *supposed* to focus on the new features of the new version, but the Dreamweaver CS3 exam was all about AP Elements and frames and other stuff that no modern developer should ever use. There wasn't, for example, a single question on Spry, which I consider to be the most significant new feature in CS3.
But that point aside ... yes, Adobe did eventually listen to us and say that instructors didn't have to take the test, but they made this announcement *one day* before the deadline to take it. So they waited until the last possible moment - until pretty much everyone who was going to have to take the test had already done so - before making the announcement. Yes, they let off the hook those who had failed the test and would therefore possibly be unable to continue teaching, but they offered nothing by way of compensation to those who may have had to take the test multiple times in order to pass it. So even this was at best a half-measure.
In addition, as you pointed out, the amnesty only applied to currently certified instructors. If I was someone who was trying to advance my career and actually *become* a certified instructor for the first time, I would have no choice to but try to pass the test. It's totally unconscionable that they are continuing to offer and market the test, and that they have done nothing to offer refunds or other compensation to those who were in essence forced to take the exam before they acknowledged the error.
I'm actually in the situation outlined above right now with other products. I am already certified in DW and Flash and a few others (although I was only able to just barely pass the DW test, and I can guarantee that it was only because of the poorly written questions), but I want to become a certified instructor for both ColdFusion and Flex. In order to do so, I will have to take and pass both of those exams. Thankfully, like you I'm now in a position where I can take them for free, so it's only a matter of my time, but I truly feel for those who do have to pay.
What's clear is that this is a deeply rooted, systemic problem at Adobe. We can now safely agree that there are three deeply flawed CS3 exams - DW, CF, and InDesign (Adobe actually pulled the ID exam and rewrote it because of the complaints.) I took and passed the Flash CS3 exam, and while it wasn't as bad as DW, it couldn't reasonably be called good by any standard. From what I can tell on their site, they currently offer exams in 10 products, so fully 25% of them are flawed, and I wouldn't be surprised to hear that the number is actually higher.
Adobe needs to decide if the certification process is something that they are going to take seriously or not. If so, then it's time for the company to decide to invest in the tests and bring it in house. Go out to the community and get actual experts to help write the tests, but keep the quality control in house. And make sure that the quality control is really there.
And by the way, I don't see criticisms like this to be attacks on Adobe, but rather well-deserved negative feedback for one particular department at Adobe that is simply not doing its job.
As someone who isn't very good at taking exams I regard them as second class qualifications against demonstratable practical experience. If you can memorised a reference manual (I think this is true for some Microsoft exams) then you should be able to pass. My memory is bad and I sometimes refer to cfQuickDocs when coding new stuff. Does that make me an unsuitable candidate for a CF job?
I wouldn't want to employ people who have a 500Gb drive plugged into their brain, I prefer creative people who can wow me with their portfolio and clever pieces of code.
If Ray got 92% then the questions need adjusting until Ray can get 100%. Otherwise what chance do the rest of us have? (Of those who really feel they need to get certified)
I'm writing a Dreamweaver CS3 exam right now for an online testing company. It's not as official or as difficult as a cert test, but it does help motivate you to learn a certain product in order to score well.
They take it seriously, using Instructional Design methods, and have hired people to review my outline plus questions. I'm working on the test outside of my regular job and the validators are contract part-time as well.
I hadn't realized how hard it would be to actually write one of these babies. Whenever I've taken a test, I can usually get through a question every 3 minutes.
But writing a question takes hours. I've devoted entire Saturdays to writing a handful of test questions (and they need 150).
They ask for beginner, Intermediate and Advanced questions, but the Advanced questions aren't to be so obscure as to be considered a trick. So someone who scores well on the test might be under the wrong impression that they aced it, when in reality it simply means they know the product pretty well.
So it more than a core competency test, but less than a certification test.
I'm definately in favor of testing. It's a time to be honest with yourself.
On a side note - did Adobe ever add a page where you can view a list of certified developers? I know right after I took my test that list disappeared from the Macromedia site and was gone for a long time. I bought Forta's cert prep book for 7 but never got around to taking it and after reading this probably won't bother with 8 either. Or maybe I should take it now while it's easy?? :)
Good post Ray..
I'm looking to take the certification exam at some point this year, simply because:
a) As a consultant, it looks good on my resume
b) I need it if I want to qualify to becomes a certified instructor, and since a lot of my POSS work with Transfer will be training, I don't really have a choice.
Now, I'm actually interested to see how badly this test really is before going forward - but this is another case where a livelihood is actually impacted by the certification.
I put off taking CF 7 certification as when I felt comfortable taking it CF 8 was launching. Certification is fairly important to me as all of my work sits inside my workplace's intranet, so unless they are already interviewing me they are never going to see the work I do.
Thanks for pointing it out, I see no point for myself to take certification now if it is dodgy... But it does make it an interesting choice for trainers, or trainers in training for taking certification at this point of time.
Good thoughts, Ray. I'm scheduled to take the CF8 ACE exam later in the month and have paid my deposit for the test but have already had second thoughts, primarily because of what I've seen in the study guide Adobe has linked on their site.
While I own a copy of the CF7 Certified Developer Study Guide and found it helpful, if rudimentary, I'm non-plussed by the expectations for this test...
I think from what i have read, the Redhat certification approach is really good, perhaps the best one out there.
The Redhat test consists of being given a broken Redhat box and the test is to get the darn thing back working again.
knowing that, i have a lot more confidence in someone with that certification
The Flex exam is also replete with errors, multiple correct answers where one is required, sentences that don't make sense, judgment calls and obvious attempts at misdirection. I scored very well in spite of this but also wrote lots of comments.
The other problems include the fact that they don't tell you what you got wrong and there's no way communicate, argue or plead with the people responsible.
I think the questions were prepared by people who couldn't cut it as lawyers, or Flex developers ;)
Cheers,
Oliver
@Oliver - You said there was no way to communicate. When I took my cert, each question, shoot, even the pre and post stuff, had a Comments box. I used it quite often. Now - I'd be willing to bet I don't hear back, but there was a mechanism at least. The Flex cert didn't have that?
@All - Thanks for the comments so far. I haven't heard officially from Adobe yet, but I'm still hoping for something.
tests are worthless... hands down, enough said. i don't know why companies rely on a test to determine whether you know a programming language or not. like you, i suck at memorizing and constantly refer to the cf reference to remember all the arguments for every tag and function out there. i've got too many other languages in my head and i'm constantly getting confused.
for the people out there that think a cf certification is worth it, let me have you talk to a certified dba i know that can't even write a stored procedure, or the certified cisco administrator that doesn't know the first thing on how to setup a router to vlan a network, or the certified windows system engineer who can't event get security permissions correct. the list goes on and on.
nothing beats experience. with the amount of open source applications you've put together over the years, your reputation and work should speak for itself. anyone who tells me that raymond camden needs to take a test to prove his jedi worthiness should be dragged into the streets and shot.
@df guy: I appreciate the vote of confidence - but again, this really isn't meant to be a 'are certs worth it in general', but more of a 'what type of product is adobe promoting'. Even if I don't put a lot of stock behind certs, I want the cert for CF to be as good as possible, know what I mean? I also feel Adobe needs to do something in regards to Dreamweaver. You can't sell what you won't stand behind.
All certifications exam are flawed. Adobe is not better or worse in this regard. Unfortunately, there is no way to defend yourself even if their answer is wrong. Just pay another $150 and give them the expected wrong answer to pass the test and get it over with.
Here's my experience from a recent Flex certification test:
http://yakovfain.javadevelo...
I have taken the old(er) cf cert, i want to say for 6, and even back then I would have to agree. I remember there were question back then that really had 2 correct answers given the right situation. Not to suggest that making the questions a story would help. More useless information would not help. I think certification should not be just a test. I personally would like to have a 2 part process. First there is a generic test, very basic programming concepts and coldfusion syntax. Then a "Solve the Real World Example" where the instructor would review with the test taker what they did and why.
just my $0.02
I've taken two CF exams, 6 and 7. Both were fine and I passed with a 96% and a 98%.
I've been waiting on the next one because I'm back in the freelance word and they certainly do help in getting work.
I understand that if you're in full time employment and intend to stay there for a while that it may seem fruitless to take the exam but what about the benefits of forcing yourself to look through documentation for functions and tags you might never normally look at?
I'm a little embarrassed saying this but it's only recently that I found out how useful Val() is! And that's after 8+ years of dev'ing.
I like exams. I like the CF exams and I intended to take the CF8 one as soon as it came out. Now I'm not so sure. I may hang back and see what happens.
Ray said: "...this really isn't meant to be a 'are certs worth it in general', but more of a 'what type of product is adobe promoting'"
I think they should start promoting ColdFusion as a product before they start promoting a certification for it. :-) Some Google AdWords would be nice.
My guess is that there's more at work here than just Adobe being bad... admittedly, with the community creating the content and someone else administering the program it's a big-time cash cow for Adobe, so I don't see them doing anything to interrupt the flow of money. On the other hand, since they're selling a faulty product to huge numbers of people, it's only a matter of time before someone spins up a class-action lawsuit against them. It's very confusing: why are they doing this?
I have been told, though, that even the certifications team has tried, internally, to create initiatives to fix it and their efforts have (some how, some way) been blocked. They don't want to have to go home ashamed of their jobs, I suspect... point is, there's more going on here than we're being told and my guess is that there's some heavy-duty legal issues behind the scenes that they CAN'T talk about. With this much chatter going on in the wild, they can't just be maintaining the status quo.
Just a fun little me too:
The adobe flex exam tends to follow the same general issues described here.. multiple right answers, poorly phrased questions and mind-reading.
When I took it, there was an answer they were 'hoping' for to an e4x question. Amusingly, the answer they were hoping to receive was actually impossible and related to an open Adobe bug on e4x processing.
That plus the fact that many of the questions would yield one answer if you had a surface knowledge of Flex and a different answer if you actually understood it, made for a wonderful experience.
I spent a fair amount of my time saying, well, I know A and B are correct, however, if I only knew a little bit about Flex, I would choose A. That strategy yielded me a great score.... and a sad disposition
I would have to agree Ray. If you are going to have a certification test, it best be done right. My issue with cert tests and most of ther online "knowledge" tets is that it looks like most of them were written not by a CF developer, but someone going through the CFWACK book looking for obscure information that the likes of you and Ben may not be fully familiar with because they are hardly used. I find that they don;t test for practical knowledge...maybe an academic wrote them that is out of touch with the real world...who knows?
The problem does come in when employers either require or rely on these tests do determine employability. I can remeber taking the Brainbench CF test for a recruiter for a consulting company. I scored well on it, but many of the questions were about things in CFML that I have never onece used in my 10 years of CF expereince. I have worked for a variety of companies/clients and developed a wide variety of apps from simple display pages to e-commecve apps. So I should have used, at least a few time, even some of the obsure stuff, but some of this was outside of my knowldege. The other thing, that others have brought up here, a measure of a developers worth is not on his abiloity to remeber every single tag/function or the details of each of those, but moreso on what tags/functions to use and where to look up the minutea when needed or where to look for what to use when needed.
I have hesitated adding to this thread but I can't do it any longer.
I have taken the DW CW3 ACE exams two times and have failed at both attempts. These two test are the only two tests that i have ever failed in my life, and I'm 46.
The tests were poorly written and contained questions on things that any professional designer/developer should never use. There were questions on outdated technology.
On the first test that I took in Feb 2008 I received the following scroes:
Understanding Web technologies - 50%
Planning sites- 72%
Designing pages - 58%
Managing and maintaining sites- 45%
My overall score was 59 with a 64 being a passing grade.
On the test that I took today I had the following scores:
Understanding Web technologies - 50%
Planning sites- 66%
Designing pages - 47%
Managing and maintaining sites- 72%
My overall score was 58 with a 64 being a passing grade.
I cannot, on good faith, recommend to anyone that they take this test.
On a lighter note, I am a frequent poster to the Adobe Dreamweaver and ColdFusion forums so if I recommend a solution to you, be aware that I am a failure!
This is to Ken Ford. Ken, you are one of the people on the list I respect. Don't say the things you said in the last line of this post. You are great. I did take the test twice and somehow passed the second time, by 2 points, so I just eeked by myself.
I took 5 this time because I have to. I am an instructor and to keep that status with Adobe I have to be certified in the classes I teach.
In order of best to worst: (keep in mind they were all recert tests, and should only cover new features)
Photoshop, InDesign (a test they reworked), Illustrator, Flash, Dreamweaver. Sad. They aren't worth anything but I have to take them.
I did take Acrobat and it was the full test. I didn't pass. It was crazy hard and had many of the same questions, just worded differently. It's not like the others. You need 76% or more to pass, you have 67 questions and 75minutes to take the test. Something like that. I'll try again though.
Dee
@Dee
It was just a joke for a sad situation, and not my sad situation. I just feel really bad for Adobe and hope that they fix the tests.
I did take the full test since I haven't taken one since DW MX 2004.
Kudos to the one maintaining this consistency.
Nothing seems to be changed. I had completed my CF 7 certification last year and could score 86 while my brother, Sagar, scored 94. We both did feel it was more of mind reading.
But I won't say the same for Flash 2004 certification that I took.
Well, I can't argue or take exception to anything that Ray states. I, too, long for the days of the ColdFusion certification program being under the wings of Allaire or Macromedia. Adobe has been a major obstacle instead of what should be viewed as a win-win relationship.
However, if you do want to or have to (job promotion, training requirement, etc...) take the CF8 certification exam, you might want to consider getting CF8 Exam Buster which I just released (http://centrasoft.com). I won't plug any further. Thanks, Brian
hi...
can someone pls send an email to me at serel.17@gmail.com and let me know how to register for coldfusion certification? ASAP please.....!!
Heaven help me, cause I know I am suppose to help people as an ACE, but honestly, was this last message a spam? @serel, forgive me, but even Ray here and most of us would have to look it up too. It's on Adobe's site and there is a search field. There is an exam certification page with all the info you need. My patience is thin today only because other intelligent people have asked similar questions to the list I subscribe to and could have done a search as easily as I can. If you typed into this thread, then you can find the answer just as easily for yourself.
I am sure I'll regret posting this later, but there ya go.
http://www.adobe.com/suppor...
http://partners.adobe.com/p...
It's been 10 months since this blog was originally posted. Does anyone know if any changes have occurred with the CF8 test since that time?
Thanks, ADS
I am not aware of any public news on this matter. I promised myself I would _not_ blog on it again. The whole topic was just too darn frustrating. But I'll probably blog the news (if any) anyway just to keep my readers up to date.
I am thinking of taking it. Need to get out of this job and would look good on my resume. I think?
The only way to ensure Adobe will look at the quality of its certifications is to send your open letter to the Adobe CEO.
I agree with Ray. I have yet to take the certification because of this. Plus, I've purchased all the certification software/PDF's/online exams I could find... and after reviewing them, I see the same issues mentioned in this blog. Here's my point of view: A Coldfusion certification should be about, get this, Coldfusion! I want to see 85% of the questions related to CF tags, CF functions, CF syntax, and CF Administrator. The balance should be database oriented. I see no value whatsoever in adding questions on HTML. After all, most of what I code in my current web output is in CSS. So if you have HTML then you need to add CSS questions. The truth of the matter is there should be NONE OF IT! Lastly, why are there questions on server architecture? Or installing Coldfusion on a server. I have worked for 9 years with Coldfusion at: a small software company (Vurv Technology), a medium sized Security department inside a big company (Lockheed Martin), and a large company supporting the DOD (General Dynamics) on classified networks. I'm now a Sr. Coldfusion Developer. I have NEVER had anything to do with server administration or network administration. They hire people just for that. Therefore, I have NEVER had to install Coldfusion on a server. Those questions are ludicrous for someone who actually does Coldfusion programming for a living. Make the exam a true Coldfusion certification by actually having real Coldfusion questions. Also, any professional in any job would know not to have created an exam full of questions/answers spelled wrong… And any teacher would know not to have a question with multiple right answers. So wake up Adobe.
So... 7 years later, how's the Coldfusion certification right now?
Don't really know or care. I mean, I care about ColdFusion, but I don't care about the certification anymore.
So, apart from the certification programs, what if someone wants to learn cfml from scratch and become an expert? Any ultimate path?
Well, I'm biased, but I'd buy the CFWACK books (the 3 for 9 plus the 1 volume update for 10).
I'd also check out "Learn CF in a Week", http://www.learncfinaweek.com/, but the server seems to be down now. It should come back though (afaik).
Ok thanks for your tips, checking it out.