Doing something a bit different today. A PhoneGap user contacted me yesterday with an interesting problem. He and I discussed it over a quick Google Hangout, and I thought I'd write up some thoughts about our discussion. Ultimately I want to build a proof of concept around this idea, but I thought I'd start first with an explanation, sans code, to see what people thought.
The problem, at a high level, involves downloading data files to a PhoneGap/Cordova application after the user has installed it. These would not be required downloads. Think more of things like DLC, or additional songs for a game, optional items. His question was how an application could be architected to support such a system. Here is what I told him. (And as always, I welcome other opinions in the comments below.)
The first thing I mentioned is that he would need some form of application server in order to handle hosting the resources. This application server needs to be able to respond to a request asking what resources are available. How the server responds isn't really important. A JSON-encoded array of URLs, with perhaps some meta information about each resource, would be sufficient. "Application Server" need not be anything complex. In fact, you could build something with Parse.com to handle the entire process. But at the end of the day you need that "responder" that can list what resources are available for download.
On the client side, you then need to be able to ask that server for a list. Obviously this is nothing more than an XHR request. You could add a layer of complexity to the process by supporting a filter. For example, if the server has 10 resources available and the client has downloaded 5 of them, maybe there is some way for the client to send that information to the server so that server only responds with what is new. Again, how complex you build this is up to you. I'd imagine that a simple list, even of 100-200 items, would transfer so fast (even on mobile) that you wouldn't need to worry about filtering.
So - you've got two parts now, server side and client side. Fetching the resources isn't terribly difficult using the File Transfer plugin. Handling a bunch of these so you don't intefere with normal application usage is another matter. In a SPA, you could simply fire off a method to do these while the user carries on their merry way. In theory then these resources just become available when they get downloaded. If you grab one at a time you can even gracefully handle the application being killed off half way through. (To be clear, I need to test that. If I download a file called foo.jpg and it doesn't finish because the application is killed, does that file exist on the file system but in a corrupt state? I'll find out!)
That's the issue at a high level. The user who spoke with me asked me to take a look at this with jQuery Mobile, and I know I'm all about AngularJS and Ionic now, but I'm going to give it a shot there first to see if I can get a proof of concept working. What about you? Have you built something like this? Do you have any advice?
Archived Comments
I have more or less built this and have ran into a few pitfalls along the way, but have a very successful platform now. Your high level description is right on target and I used jQuery Mobile as well for a few of the interface items.
Parse worked great for a bit, but only could manage files up to 10 MB, so a show stopper there. Also, it wasn't that great of an interface for clients that were not "tech savvy". I could continue on - if there are other questions that come up, i can try to answer them.
I keep thinking it would be SO nice if Jack Kelly were to port his LoaderMax library to JavaScript and Cordova, as he did GSAP, giving us a nice API for queueing and loading files of various types, with progress tracking and events for each queue/file.
I'll say that I've built similar mechanisms on platforms that are more and less agreeable that iOS. The real challenge is if you require actual transactions (cards, paypal, bitcoin, pick your poison(s)). Because, you'd then need to manage transaction state, and you can no longer trust the client. You can reasonably trust the client if you deal in entirely virtual money/rewards because you don't mind cheating. Going back to the stated problem, there plenty of great ways to handle things. You've got a local db, file-system and cookies at your disposal.
I will also build a system like this and it will be interesting to follow your ideas here.
What I also planned was a tag to give version numbers to files in case I need to update them. So when the application checks if the file exists it will check also if the version number is smaller then the one listed in the json file.
I just recently built in this same method you have mentioned in an Ionic / Angular JS app that I am building (talking to a REST API). Am using ngCordova too to for the file download functions. Does help. and its pretty straight forward. I do not use ngCordova fully though bcoz I dont find a way to use 'dataDirectory', 'applicationDirectory' etc in it. It still refers to localFileSystem.PERSISTENT. Do suggest if I should still use ngCordova fully.
@Amrudesh: Well, I think your last question is a bit OT for this blog post, but, the short answer is that the best thing to do is simply file a bug report with the ngCordova project. You can do so here: https://github.com/driftyco...
@Raymond . Ya. True. Could have avoided that. Just got carried away. And yes, I have also filed that bug report. Thnx.
I've actually started working on a design for something very similar. We need the ability to update or add certain images to our app. They won't change very often but often enough that we want to be able to host the images on the server and when there is a new or updated image the app downloads it and stores it in the file system. I was in fact looking at the example you posted here as a model: http://www.adobe.com/devnet...
I think what we're going to do is make the API resource able to accept some parameters like lastModifiedDate. The resource will then zip up whatever images have been added or modified since that date and sent back in the response. If no parameters are provided it will include all of the images. What I really like about your example was the ability to make a HEAD request to check the Last-Modifed date of the resource and let the client determine if it is already up-to-date or if it needs to request any new/updated resources.
@Erin: Re that article: I considered mentioning that, but I haven't tested the library under Cordova. Not sure how well it would perform.
Great point Ray, I often find it challenging, without actually testing it out first, to decide when it's better to go with multiple smaller separate requests versus one big one.
@Tobias: What is your plan for where you were going to store the version number for each file?
I also originally was thinking of using a version for files in my design and then decided to go with checking the last modified time instead. I figured if I can just use the file system API to give me metadata about the files I already have then I can avoid using a database on the client to store this information. But I'm having trouble figuring out what all is included in the Metadata object. All I can find is the modificationTime property. It would be great if we could add other properties when we store files.
@Erin: Your idea with the last modified date is quite good. But I wanted to start really slow and not develop a sever that has all the logic needed to make the selection of modified files since $date.
My idea was, to have something small working without much work. To edit the json list myself and change the version number as I replace the files in the folder.
Then the app would check its internal list (in localstorage) and compare the version numbers with the ones in the json list. -> then download everything that is smaller and replace + set the new version number.
Well, as I said it's quite a manual approach at the beginning. I'll have to develop some kind of server that can manage version numbers on its own to become more efficient.