Quick Example - Parsing USGS Earthquake Data

This post is more than 2 years old.

Apparently there were a few earthquakes yesterday. All I know is that I'm happy I live in a state where our natural disasters are huge and slow moving. (Not that I left last time a hurricane hit anyway.) One of the cool services the US Geological Service provides is a set of feeds for earthquake data. These are organized by how far back they go in time as well as how low of a magnitude they include. Since there's quite a few small earthquakes every day this helps you focus on more of the "OMG THE WORLD IS ENDING" variety quake as opposed to the "Did the garbage truck just go by" mini-quake. The USGS provides both RSS feeds and CSV, and I thought it would be cool to whip up a quick ColdFusion demo that makes use of this data stream.

I began by looking at the RSS feeds. I know ColdFusion makes RSS parsing easy and in general CSV just feels messy. But I noticed something odd. None of the RSS feeds provided the magnitude of the earthquake in a "data safe" format. What do I mean? Well consider this snippet:

<entry> <id>urn:earthquake-usgs-gov:us:b0006b3y</id> <title>M 4.9, Tonga</title> <updated>2011-10-21T14:56:05Z</updated> <link rel="alternate" type="text/html" href="http://earthquake.usgs.gov/earthquakes/recenteqsww/Quakes/usb0006b3y.php"/> <link rel="related" type="application/cap+xml" href="http://earthquake.usgs.gov/earthquakes/catalogs/cap/usb0006b3y" /> <summary type="html"> <![CDATA[ <img src="http://earthquake.usgs.gov/images/globes/-20_-175.jpg" alt="21.107°S 175.602°W" align="left" hspace="20" /> <p>Friday, October 21, 2011 14:56:05 UTC<br> Saturday, October 22, 2011 03:56:05 AM at epicenter</p> <p><strong>Depth</strong>: 191.60 km (119.05 mi)</p> ]]> </summary> <georss:point>-21.1065 -175.6015</georss:point> <georss:elev>-191600</georss:elev> <category label="Age" term="Past day"/> </entry>

While you can see the mangitude in the title, and it is pretty consistent, I didn't like the idea of regexing it out just to get the value. So I took a look at the CSV data instead. Here's a sample from their feed:

Src,Eqid,Version,Datetime,Lat,Lon,Magnitude,Depth,NST,Region us,b0006b3y,3,"Friday, October 21, 2011 14:56:05 UTC",-21.1065,-175.6015,4.9,191.60,17,"Tonga"

Perfect! Not only is this exactly what I want - it's actually a lot slimmer than the XML. But how will I parse it? I'm going to use an old feature - one I haven't used in years. Did you know that CFHTTP automatically supports parsing CSV type files? Check it out:

<!--- This URL represents M2.5+ quakes over the past day ---> <cfset m25oneday = "http://earthquake.usgs.gov/earthquakes/catalogs/eqs1day-M2.5.txt">

<!--- get the URL and turn it into a query ---> <cfhttp url="#m25oneday#" method="get" firstrowasheaders="true" name="items">

Do I need to parse the CSV? Not at all. Here's how the dump looks:

Wow, I'm done. So I whipped up a small little application that let's me use jQuery to click for a map detail of the earthquake epicenter. This makes use of the super-simple static Google Map API. Here's the complete source:

<!--- Source: http://earthquake.usgs.gov/earthquakes/catalogs/ --->

<!--- This URL represents M2.5+ quakes over the past day ---> <cfset m25oneday = "http://earthquake.usgs.gov/earthquakes/catalogs/eqs1day-M2.5.txt">

<!--- get the URL and turn it into a query ---> <cfhttp url="#m25oneday#" method="get" firstrowasheaders="true" name="items">

<html> <head> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script> <script> $(document).ready(function() { var mapImg = $("#quakeMap");

$("div.quakeItem").click(function(e) {
	var lat = $(this).data("lat");
	var lon = $(this).data("lon");
	mapImg.attr("src","http://maps.googleapis.com/maps/api/staticmap?center="+lat+","+lon+"&zoom=7&sensor=false&size=400x400");
	console.log(mapImg.attr("src"));
});

}) </script> <style> #quakeMap { float:right; } div.quakeItem { cursor:hand; } </style> </head>

<body> <h2>M2.5+ Earthquakes</h2>

<img id="quakeMap" >

<cfoutput query="items"> <div class="quakeItem" data-lat="#lat#" data-lon="#lon#"> <h3>#region#</h3> <p> A #magnitude# earthquake at #datetime# </p> </div> </cfoutput>

</body> </html>

So how does it work? I simply loop over my query and output the data. Notice I make use of data attributes to store my latitude and longitude. This is one of my favorite HTML5 features. If you look up at the JavaScript code you can see how easy it is to fetch that data out. Once I have it I simply update my image to point to Google's API. And I'm done! Try the demo below. (And yes, I removed the console.log message.)

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA https://www.raymondcamden.com

Archived Comments

Comment 1 by John Sieber posted on 10/21/2011 at 9:05 PM

Great post, it is always fun to see different ways to work with all of the data that is out in the wild.I was a little confused in the first paragraph when you mentioned that they have both rss and css data feeds. It made more sense once I realized you meant csv. Thanks for another cool demo!

Comment 2 by Raymond Camden posted on 10/21/2011 at 10:11 PM

Oops - type fixed John - thanks!

Comment 3 by Tim Barber posted on 10/21/2011 at 11:13 PM

Wow, I really love this but I am surprised you didn't use CFPDF in there anywhere. At work we created an entire framework based on CFPDF and found it to be invaluable. Life hasn't been the same for us since Adobe provided us with CFPDF, that's for sure!

Comment 4 by Raymond Camden posted on 10/21/2011 at 11:16 PM

Um.... ok. You like PDFs, eh? :)

Comment 5 by Tim Barber posted on 10/22/2011 at 12:25 AM

Do I like PDF's? Hey I'm a CF developer, aren't I?! LOL!! I have a bumper sticker that says "Long Live CFPDF!" LOLOL!

Comment 6 by Indy Griffiths posted on 10/22/2011 at 1:26 AM

I made something very similar a couple months ago but from GeoNet instead of the USGS. I don't have a production CF sever, but you can grab the code here: http://www.indygriffiths.co...

Go easy, this is one of the first things I made in CF!

Comment 7 by Topper Harley posted on 10/22/2011 at 2:00 AM

Indy,

That code is the nicest code this blog has ever seen. You're new? Get out of ColdFusion now while you still have a chance!

Comment 8 by Tim Barber posted on 10/22/2011 at 2:03 AM

I don't know Topper's point, but personally I also enjoyed that code quite a bit Indy. It would have been nice to see more use of CFPDF though. Is that something you are looking at adding?

Comment 9 by Phillip Senn posted on 10/22/2011 at 7:05 PM

Here's an animation showing the day of the Fukushima quake.
http://www.japanquakemap.com/
You'll want to watch it past 14:46 when the big one hit.

Comment 10 by Raymond Camden posted on 10/22/2011 at 7:19 PM

My god - that's insane.

Comment 11 by Phillip Senn posted on 10/22/2011 at 7:24 PM

Yeah, and if you view source, it looks like it's fairly straightforward.

Comment 12 by Raymond Camden posted on 10/22/2011 at 8:47 PM

Ignore Topper. He seems to troll here once every few months or so. I'd delete it - but it's easier to let ignorance shine. ;)

Comment 13 by Pablo Vos posted on 10/22/2011 at 11:13 PM

Thanks for this Raymond. Will put this to good use!

Comment 14 by Don Vawter posted on 10/24/2011 at 9:08 PM

Now just add the code to reposition the top of the image to the position of the entry you clicked and you won't have to scroll back to the top to see it.

Comment 15 by Raymond Camden posted on 10/24/2011 at 9:32 PM

Hmm. I haven't done a lot of prepend/insert/etc in jQuery like that. It's simple - but would be a good exercise.

Comment 16 by Raymond Camden posted on 10/24/2011 at 9:40 PM

Thank you for the inspiration Don. I posted it here:

http://www.coldfusionjedi.c...

View source to see the new client-side code. Server-side code is the same.

Comment 17 by Don Vawter posted on 10/24/2011 at 9:54 PM

Now you are cooking with gas!

Comment 18 by Shaun posted on 11/22/2011 at 11:47 PM

If you thought the Japanese quake map was insane, try the original version based on the earthquakes in Christchurch New Zealand.
http://www.christchurchquak...

9079 earthquakes in the region thus far

Comment 19 by Raymond Camden posted on 11/23/2011 at 12:10 AM

Impressive - although this is not a fight I'd want to win. ;)

Comment 20 by Henry posted on 12/1/2011 at 5:32 PM

So Ray,
I have been playing around with this example a bit and seem to have found a bug in cfhttp. I am reading from this link: https://docs.google.com/spr.... (csv) and this, so you can view in your browser.
https://docs.google.com/spr...

Using cfhttp I can view the result with cfhttp.filecontent but when I add the "name" attribute to cfhttp to generate a query, an error is thrown. "Incorrect number of columns in row". I have tested your work above and generate the same results you have. I have seen other posts related to this but no fixes. Any ideas or workarounds.
Thanks
Henry

Comment 21 by Raymond Camden posted on 12/1/2011 at 5:44 PM

Your first link doesn't work for me. Can you put the CSV file someplace else?

It sounds like the data is bad. CF is assuming the same name of cols per row and if the data doesn't match that, then I can see it being upset.

Comment 22 by Raymond Camden posted on 12/1/2011 at 5:45 PM

Sorry, the link DID work - I didn't see the download.

Comment 23 by Raymond Camden posted on 12/1/2011 at 5:49 PM

Yeah, I'm thinking I'm right - it's not happy with the data. So just parse it yourself. Use cfhttp to get the data (don't use the name) and just parse it manually.

Comment 24 by Henry posted on 12/1/2011 at 5:52 PM

The first link probably downloaded as csv so you might want to remove from your local system, if it did. You can read the second link in your browser.

I found this on Adobe Forums and is a fix.
http://forums.adobe.com/mes...

Thanks

Comment 25 by Raymond Camden posted on 12/1/2011 at 7:26 PM

Um, yes, that's what I said. ;) Download the data raw data and process it manually. :)

Comment 26 by Henry posted on 12/1/2011 at 7:38 PM

Yea I thought that's what you said but wanted to double check! Oh well, so much for other plans.
Thanks :-{(