Handling JSON with prefixes in jQuery and jQueryUI

This post is more than 2 years old.

William asked:

How do you tell jquery in an autocomplete to remove the leading // from the json response? Do you need to use an event function to manually do it, or is there a simple way? This prefixing of JSON responses is mandatory for one of my projects.

Yes, you do need an event function to manually do it, but luckily jQuery makes this rather simple. To test this, I began with a very simple autocomplete implementation. This example uses ColdFusion on the back end but obviously it would work with any application server. First, here is the front end:

<html>

&lt;head&gt;
&lt;script type="text/javascript" src="http://code.jquery.com/jquery-1.7.min.js"&gt;&lt;/script&gt;
&lt;script src="jquery-ui-1.8.16.custom/js/jquery-ui-1.8.16.custom.min.js"&gt;&lt;/script&gt;
&lt;link rel="stylesheet" href="jquery-ui-1.8.16.custom/css/vader/jquery-ui-1.8.16.custom.css" type="text/css"&gt;
&lt;script&gt;
$(function() {

	$("#name").autocomplete({
		source: "source.cfc?method=searchart&returnformat=json"
	});

});
&lt;/script&gt;
&lt;/head&gt;

&lt;body&gt;
	
&lt;form action="" method="post"&gt;	
name: &lt;input name="name" id="name" /&gt;
&lt;/form&gt;

&lt;/body&gt;

</html>

And here is the back end. Again - this could be any language at all.

component {

remote function searchArt(string term) {
	var q = new com.adobe.coldfusion.query();
	q.setSQL("select artname from art where lcase(artname) like :term");
	q.addParam(name="term",value="%#lcase(arguments.term)#%",cfsqltype="cf_sql_varchar");
	var query = q.execute().getResult();
	var result = [];
	for(var i=1; i&lt;=query.recordCount; i++) {
		result[arrayLen(result)+1] = query.artname[i];
	}
	return result;	
	
}

}

This example makes use of jQuery UI's Autocomplete controls, one of my favorite parts of jQuery UI. As you can see it is rather simple to use. I simply tell it which input field I want to turn into an autocomplete and tell it how to fetch the data. (There are many more options but for now, this is all we need.) This runs fine until we begin to use the feature William spoke of - prefixing the JSON result with two / characters in front. This is used in some situations to help make the Ajax application a bit more secure. By prefixing the JSON string we require preprocessing on the string before blindly evaluating it. While how this is done will depend on your application server, in ColdFusion you can turn this on at the server level or on a per application basis. Here's how I did it for this demo:

component {

this.name="autocompleteprefixissue";
this.datasource="cfartgallery";
this.secureJSON="true";
this.secureJSONPrefix="//";

}

Once this feature is enabled, your autocomplete stops working. You get no error. You're really stuck. If you know enough to check the network monitor, you can see the issue:

So how do you fix it? One of the more cooler features of jQuery is the ability to register global Ajax handlers. But we're going to tackle this a bit differently and instead use ajaxSetup. This is similar but works at a slightly lower level. One of the features is called dataFilter. As you can guess, it allows you to filter the data returned by Ajax requests. What's cool is that this happens behind the scenes. Our autocomplete code will never know it even occurred. Let's look at how it works.

<html>

&lt;head&gt;
&lt;script type="text/javascript" src="http://code.jquery.com/jquery-1.7.min.js"&gt;&lt;/script&gt;
&lt;script src="jquery-ui-1.8.16.custom/js/jquery-ui-1.8.16.custom.min.js"&gt;&lt;/script&gt;
&lt;link rel="stylesheet" href="jquery-ui-1.8.16.custom/css/vader/jquery-ui-1.8.16.custom.css" type="text/css"&gt;
&lt;script&gt;
$(function() {

	$.ajaxSetup({
		dataFilter: function(data, type){
			return data.substring(2, data.length);
		}
	});

	$("#name").autocomplete({
		source: "source.cfc?method=searchart&returnformat=json"
	});

});
&lt;/script&gt;
&lt;/head&gt;

&lt;body&gt;
	
&lt;form action="test4.cfm" method="post"&gt;	
name: &lt;input name="name" id="name" /&gt;
&lt;/form&gt;

&lt;/body&gt;

</html>

My dataFilter function takes in the data stream and simply removes the first two characters. As I said, nothing in autocomplete was changed. As soon as this is added, things begin working perfectly again. Try this yourself below (and look - I didn't put any pesky console.log messages so all your poor folks in older IE builds or Firefox w/o Firebug can enjoy it!).

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA https://www.raymondcamden.com

Archived Comments

Comment 1 by matt posted on 11/9/2011 at 8:07 AM

Maybe this is the wrong way to go about it, but you can turn off JSON prefixes in the CF admin, wouldn't that achieve the same result?

Comment 2 by Raymond Camden posted on 11/9/2011 at 8:33 AM

Right, but in this case, the prefix is out of the end user's control. (I don't think he is even using ColdFusion.)

Comment 3 by Lola LB posted on 11/9/2011 at 5:12 PM

There's a missing file -

File not found: /demos/2011/nov/8/test4.cfm

Comment 4 by Raymond Camden posted on 11/9/2011 at 5:27 PM

Where do you see that link?

Comment 5 by Raymond Camden posted on 11/9/2011 at 5:28 PM

Oh, you submitted the form. Don't do that. ;) I removed it from the action in the source above - not from the demo.

Comment 6 by Jon Briccetti posted on 11/11/2011 at 9:43 PM

Great info - can you explain what the JSON has to be prefixed - I've heard about doing this and have similar requirements with some clients. It makes sense in a way that the data feed can't just be thrown on the right hand side of an equals sign and "executed" as an expression; that could make the browser vulnerable to an exploit if a nasty expression got injected (if the Ajax call got hijacked) So it seems to me that the setup function should do more than just remove the prefix, but should also (first) validate the data is 1) from where it is supposed to be and 2) is structurally sound. but I'm speculating. Any thoughts on this?

Comment 7 by Raymond Camden posted on 11/11/2011 at 10:17 PM

I'm not sure if you _could_ detect a network hijack of some sort. It would seem impossible to me!

Comment 8 by Shaun Byrnes posted on 7/11/2012 at 2:06 PM

Hi Ray,

Good post. This is probably slightly better code to use:

$.ajaxSetup({
dataFilter: function(data, type){
return type == 'json' ? data.replace(/^(\/{2})?/, '') : data;
}
});

Just to ensure that other response types are not messed with and the // is only removed if it exists - otherwise if that setting was updated for whatever reason it would removed part of the real response.

-s

Comment 9 by Raymond Camden posted on 7/11/2012 at 5:42 PM

@Shaun: Nice! Yeah, I agree that makes more sense.

Comment 10 by Shaun Byrnes posted on 7/12/2012 at 3:37 AM

No problem

Comment 11 by Steve posted on 3/19/2013 at 8:08 PM

I just started wrestling with this today as I'm working on converting a large site from CF 8 to CF 10. Where my code is all CF I've got my modifications in place. However where it's jQuery with the validate plugin I'm having a problem. I can't see any indication that the ajaxSetup() is doing anything to the response. I put the relevant jQuery code up at pastebin (http://pastebin.com/Gu21uQZ8). I'm guessing I'm going about this completely wrong?

Comment 12 by Raymond Camden posted on 3/20/2013 at 5:41 AM

As far as I know, ajaxSetUp is *global*, so every plugin that uses XHR should use it. Your code looks right to me. Is this online where I can run it?

Comment 13 by Raymond Camden posted on 3/20/2013 at 5:44 AM

I just found a blog post that indicated that jQuery Validate may break ajaxSetup. I'm trying to confirm now.

Comment 14 by Raymond Camden posted on 3/20/2013 at 5:49 AM

I think it may be the validate plugin. Try copying dataFilter argument to the remote block.

Comment 15 by GEH posted on 4/24/2013 at 12:52 AM

Thank you, thank you, thank you! While I did need the information about the json prefix (thank you!), what really saved my day was the comment about the network monitor. I've been using the Chrome developer resource for years, but never drilled far enough down in that panel to see that it returned the information I needed.

I had been having trouble with a CFC that was being called with jQuery and was supposed to reurn json, and it worked fine on my local system, but wasn't working at all on the remote server. I had written debug info into the CFC code at strategic spots to be written to a file, but the file wasn't being written either, and I was flying blind and getting very frustrated.

But after seeing your reference to the network monitor I was able to have the component return a comprehensive error message in that window. Thank you SO much! You have saved me from hours of hair pulling. You're awesome. Have a great day!

Comment 16 by Raymond Camden posted on 4/24/2013 at 1:04 AM

Glad it helped. Check out this other resource for learning about DevTools: http://www.raymondcamden.co...