Cordova and Large Asset Downloads - An Abstract

This post is more than 2 years old.

Doing something a bit different today. A PhoneGap user contacted me yesterday with an interesting problem. He and I discussed it over a quick Google Hangout, and I thought I'd write up some thoughts about our discussion. Ultimately I want to build a proof of concept around this idea, but I thought I'd start first with an explanation, sans code, to see what people thought.

The problem, at a high level, involves downloading data files to a PhoneGap/Cordova application after the user has installed it. These would not be required downloads. Think more of things like DLC, or additional songs for a game, optional items. His question was how an application could be architected to support such a system. Here is what I told him. (And as always, I welcome other opinions in the comments below.)

The first thing I mentioned is that he would need some form of application server in order to handle hosting the resources. This application server needs to be able to respond to a request asking what resources are available. How the server responds isn't really important. A JSON-encoded array of URLs, with perhaps some meta information about each resource, would be sufficient. "Application Server" need not be anything complex. In fact, you could build something with Parse.com to handle the entire process. But at the end of the day you need that "responder" that can list what resources are available for download.

On the client side, you then need to be able to ask that server for a list. Obviously this is nothing more than an XHR request. You could add a layer of complexity to the process by supporting a filter. For example, if the server has 10 resources available and the client has downloaded 5 of them, maybe there is some way for the client to send that information to the server so that server only responds with what is new. Again, how complex you build this is up to you. I'd imagine that a simple list, even of 100-200 items, would transfer so fast (even on mobile) that you wouldn't need to worry about filtering.

So - you've got two parts now, server side and client side. Fetching the resources isn't terribly difficult using the File Transfer plugin. Handling a bunch of these so you don't intefere with normal application usage is another matter. In a SPA, you could simply fire off a method to do these while the user carries on their merry way. In theory then these resources just become available when they get downloaded. If you grab one at a time you can even gracefully handle the application being killed off half way through. (To be clear, I need to test that. If I download a file called foo.jpg and it doesn't finish because the application is killed, does that file exist on the file system but in a corrupt state? I'll find out!)

That's the issue at a high level. The user who spoke with me asked me to take a look at this with jQuery Mobile, and I know I'm all about AngularJS and Ionic now, but I'm going to give it a shot there first to see if I can get a proof of concept working. What about you? Have you built something like this? Do you have any advice?

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA https://www.raymondcamden.com

Archived Comments

Comment 1 by Greg Hudy posted on 10/8/2014 at 1:41 AM

I have more or less built this and have ran into a few pitfalls along the way, but have a very successful platform now. Your high level description is right on target and I used jQuery Mobile as well for a few of the interface items.
Parse worked great for a bit, but only could manage files up to 10 MB, so a show stopper there. Also, it wasn't that great of an interface for clients that were not "tech savvy". I could continue on - if there are other questions that come up, i can try to answer them.

Comment 2 by Nick Collins posted on 10/8/2014 at 2:28 AM

I keep thinking it would be SO nice if Jack Kelly were to port his LoaderMax library to JavaScript and Cordova, as he did GSAP, giving us a nice API for queueing and loading files of various types, with progress tracking and events for each queue/file.

Comment 3 by Rob Colburn posted on 10/8/2014 at 5:31 AM

I'll say that I've built similar mechanisms on platforms that are more and less agreeable that iOS. The real challenge is if you require actual transactions (cards, paypal, bitcoin, pick your poison(s)). Because, you'd then need to manage transaction state, and you can no longer trust the client. You can reasonably trust the client if you deal in entirely virtual money/rewards because you don't mind cheating. Going back to the stated problem, there plenty of great ways to handle things. You've got a local db, file-system and cookies at your disposal.

Comment 4 by Tobias Kausch posted on 10/8/2014 at 11:35 AM

I will also build a system like this and it will be interesting to follow your ideas here.
What I also planned was a tag to give version numbers to files in case I need to update them. So when the application checks if the file exists it will check also if the version number is smaller then the one listed in the json file.

Comment 5 by Amrudesh Santhanam posted on 10/8/2014 at 3:11 PM

I just recently built in this same method you have mentioned in an Ionic / Angular JS app that I am building (talking to a REST API). Am using ngCordova too to for the file download functions. Does help. and its pretty straight forward. I do not use ngCordova fully though bcoz I dont find a way to use 'dataDirectory', 'applicationDirectory' etc in it. It still refers to localFileSystem.PERSISTENT. Do suggest if I should still use ngCordova fully.

Comment 6 by Raymond Camden posted on 10/8/2014 at 3:13 PM

@Amrudesh: Well, I think your last question is a bit OT for this blog post, but, the short answer is that the best thing to do is simply file a bug report with the ngCordova project. You can do so here: https://github.com/driftyco...

Comment 7 by Amrudesh Santhanam posted on 10/8/2014 at 3:23 PM

@Raymond . Ya. True. Could have avoided that. Just got carried away. And yes, I have also filed that bug report. Thnx.

Comment 8 by Erin Doyle posted on 10/8/2014 at 5:04 PM

I've actually started working on a design for something very similar. We need the ability to update or add certain images to our app. They won't change very often but often enough that we want to be able to host the images on the server and when there is a new or updated image the app downloads it and stores it in the file system. I was in fact looking at the example you posted here as a model: http://www.adobe.com/devnet...

I think what we're going to do is make the API resource able to accept some parameters like lastModifiedDate. The resource will then zip up whatever images have been added or modified since that date and sent back in the response. If no parameters are provided it will include all of the images. What I really like about your example was the ability to make a HEAD request to check the Last-Modifed date of the resource and let the client determine if it is already up-to-date or if it needs to request any new/updated resources.

Comment 9 by Raymond Camden posted on 10/8/2014 at 6:23 PM

@Erin: Re that article: I considered mentioning that, but I haven't tested the library under Cordova. Not sure how well it would perform.

Comment 10 by Erin Doyle posted on 10/8/2014 at 7:34 PM

Great point Ray, I often find it challenging, without actually testing it out first, to decide when it's better to go with multiple smaller separate requests versus one big one.

Comment 11 by Erin Doyle posted on 10/8/2014 at 7:43 PM

@Tobias: What is your plan for where you were going to store the version number for each file?

I also originally was thinking of using a version for files in my design and then decided to go with checking the last modified time instead. I figured if I can just use the file system API to give me metadata about the files I already have then I can avoid using a database on the client to store this information. But I'm having trouble figuring out what all is included in the Metadata object. All I can find is the modificationTime property. It would be great if we could add other properties when we store files.

Comment 12 by Tobias Kausch posted on 10/8/2014 at 7:56 PM

@Erin: Your idea with the last modified date is quite good. But I wanted to start really slow and not develop a sever that has all the logic needed to make the selection of modified files since $date.

My idea was, to have something small working without much work. To edit the json list myself and change the version number as I replace the files in the folder.
Then the app would check its internal list (in localstorage) and compare the version numbers with the ones in the json list. -> then download everything that is smaller and replace + set the new version number.

Well, as I said it's quite a manual approach at the beginning. I'll have to develop some kind of server that can manage version numbers on its own to become more efficient.