Posted in Mobile | Posted on 01-19-2012 | 5,838 views
For the past week or so I've been looking at file system access and downloads with PhoneGap. Before going any further, I want to warn folks that I'm still a bit fuzzy on the details here. It was a bit of a struggle to get this working right, and I plan to follow this entry up with a look at iOS and also how to get all platforms working right, but for now, consider this a first draft. I also want to give thanks to Simon Mac Donald for his help. Anything right here is thanks to him and anything wrong is my fault.
Ok, with that out of the way. Let's talk about file downloads. A reader pinged me recently to ask about how to support offline PhoneGap applications. Specifically, he wanted to work with images that were remote and make them available to the application when the device was offline. I decided to work on a simple application that would fetch images from a server and store them locally.
I began by looking over the File docs at PhoneGap. This is - for the most part - a wrapper for the W3C File API. I had a real hard time grokking this API. My gut take on it is this:
- You begin by requesting a file system. This request is either for a persistent or temporary storage. Obviously which you pick depends on what your needs are. For my demo application, I need the persistent storage.
- What you get back is a file system object. From what I see in the spec, the object contains a few properties, but your primary usage of this is to get a directory entry.
- Once you have a directory object, you can enumerate files, read them, whatever.
Based on what I learned from Simon, in Android, the place you want to store your files is:
Android/data/X
Where X is the identify of your application. For my demo, this was com.camden.imagedownloaddemo. For the first iteration of my demo, I requested the file system, the directory, and then a list of files:
2<html>
3<head>
4<meta name="viewport" content="width=320; user-scalable=no" />
5<meta http-equiv="Content-type" content="text/html; charset=utf-8">
6<title>Image Download Demo</title>
7
8<script type="text/javascript" charset="utf-8" src="phonegap-1.3.0.js"></script>
9<script type="text/javascript" charset="utf-8">
10//Global instance of DirectoryEntry for our data
11var DATADIR;
12
13//Loaded my file system, now let's get a directory entry for where I'll store my crap
14function onFSSuccess(fileSystem) {
15 fileSystem.root.getDirectory("Android/data/com.camden.imagedownloaddemo",{create:true},gotDir,onError);
16}
17
18//The directory entry callback
19function gotDir(d){
20 DATADIR = d;
21 var reader = DATADIR.createReader();
22 reader.readEntries(gotFiles,onError);
23}
24
25//Result of reading my directory
26function gotFiles(entries) {
27 console.log("The dir has "+entries.length+" entries.");
28 for (var i=0; i<entries.length; i++) {
29 console.log(entries[i].name+' '+entries[i].isDirectory);
30 }
31}
32
33function onError(e){
34 console.log("ERROR");
35 console.log(JSON.stringify(e));
36}
37
38function onDeviceReady() {
39 window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, onFSSuccess, null);
40}
41
42function init() {
43document.addEventListener("deviceready", onDeviceReady, true);
44}
45</script>
46
47</head>
48<body onload="init();" >
49<h2>Image Download Demo</h2>
50
51<div id="status"></div>
52
53</body>
54</html>
As everything is async, the code gets a bit complex, but I begin by requesting the file system, requesting the directory (and notice, you can pass an optional argument to automatically create it, which is useful), and then the files.
Ok - so that seemed to work. It was then time to look into the file sync aspects. To keep things simple, my sync logic would just ask a remote server for a list of images. Every image the remote server had that I did not, I downloaded. Obviously this means I can be left with images locally I don'rt need, but I wanted to keep things as basic as possible. Here's the new version:
2<html>
3<head>
4<meta name="viewport" content="width=320; user-scalable=no" />
5<meta http-equiv="Content-type" content="text/html; charset=utf-8">
6<title>Image Download Demo</title>
7<script type="text/javascript" charset="utf-8" src="jquery.min.js"></script>
8<script type="text/javascript" charset="utf-8" src="phonegap-1.3.0.js"></script>
9<script type="text/javascript" charset="utf-8">
10//Global instance of DirectoryEntry for our data
11var DATADIR;
12var knownfiles = [];
13
14//Loaded my file system, now let's get a directory entry for where I'll store my crap
15function onFSSuccess(fileSystem) {
16 fileSystem.root.getDirectory("Android/data/com.camden.imagedownloaddemo",{create:true},gotDir,onError);
17}
18
19//The directory entry callback
20function gotDir(d){
21 console.log("got dir");
22 DATADIR = d;
23 var reader = DATADIR.createReader();
24 reader.readEntries(function(d){
25 gotFiles(d);
26 appReady();
27 },onError);
28}
29
30//Result of reading my directory
31function gotFiles(entries) {
32 console.log("The dir has "+entries.length+" entries.");
33 for (var i=0; i<entries.length; i++) {
34 console.log(entries[i].name+' dir? '+entries[i].isDirectory);
35 knownfiles.push(entries[i].name);
36 renderPicture(entries[i].fullPath);
37 }
38}
39
40function renderPicture(path){
41 $("#photos").append("<img src='file://"+path+"'>");
42 console.log("<img src='file://"+path+"'>");
43}
44
45function onError(e){
46 console.log("ERROR");
47 console.log(JSON.stringify(e));
48}
49
50function onDeviceReady() {
51 //what do we have in cache already?
52 $("#status").html("Checking your local cache....");
53 window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, onFSSuccess, null);
54}
55
56function appReady(){
57 $("#status").html("Ready to check remote files...");
58 $.get("http://www.raymondcamden.com/demos/2012/jan/17/imagelister.cfc?method=listimages", {}, function(res) {
59 if (res.length > 0) {
60 $("#status").html("Going to sync some images...");
61 for (var i = 0; i < res.length; i++) {
62 if (knownfiles.indexOf(res[i]) == -1) {
63 console.log("need to download " + res[i]);
64 var ft = new FileTransfer();
65 var dlPath = DATADIR.fullPath + "/" + res[i];
66 console.log("downloading crap to " + dlPath);
67 ft.download("http://www.raymondcamden.com/demos/2012/jan/17/" + escape(res[i]), dlPath, function(){
68 renderPicture(dlPath);
69 console.log("Successful download");
70 }, onError);
71 }
72 }
73 }
74 $("#status").html("");
75 }, "json");
76
77}
78
79function init() {
80document.addEventListener("deviceready", onDeviceReady, true);
81}
82</script>
83<style>
84img {
85 max-width: 200px;
86}
87</style>
88</head>
89<body onload="init();" >
90<h2>Image Download Demo</h2>
91
92<div id="status"></div>
93
94<div id="photos"></div>
95
96</body>
97</html>
Ok, it's a bit much, but let's work through the various events. You can still see the file system request as well as the directory list. I do two new things now once I have the files. I remember them (storing them in knownfiles), and I render them using a simple utility function. Yes - you can pass a path to an image source and it works just fine.
Now - take a look at appReady. This handles my remote call. I'll share the ColdFusion code if folks want, but all it's doing is returning a JSON-encoded array of images. For each result, I see if I already have it, and if not, use the download method of the FileTransfer object. Note: One of my images had a space in the file name. This causes all kinds of problems until I simply escaped it:
Here's a quick screen shot. Obviously it is static so you can't see it working, but in my testing, when I pushed up a new image remotely, and reran the application, it immediately noticed it was missing one and grabbed it.
So - what's next? As I said, this is currently Android specific, and that's bad. I'm next going to test on iOS, and then get one application that can handle both. Also, I didn't actually bother checking to see if the device was online. That would be trivial via the Connection API and should be done. (I'll remember to do it for the final, "combined" demo.)
Does this make sense? Any questions?
Edit on January 20, 2012: Note that I made a mistake in my fileTransfer callback. I talk about this mistake here, but the critical fix is right here:
2console.log("downloading crap to " + dlPath);
3ft.download("http://www.raymondcamden.com/demos/2012/jan/17/" + escape(res[i]), dlPath, function(e){
4 renderPicture(e.fullPath);
5 console.log("Successful download of "+e.fullPath);
6}, onError);
M.
I'm downloading all the images on app load and then replacing the src of the image on page load.
/* change images to local on page load */
function renderLocalImage(){
var contentObj = $("#pageContent");
$('img',contentObj).each(function(index){
var src = $(this).attr('src');
var srcArray = src.split('/');
var imageName = srcArray[srcArray.length-1];
var localImage = 'file://' + imageDir.fullPath + '/' + imageName;
$(this).attr('src',localImage);
});
}
Also, on ICS (4.0.3) I can't even get open database to work. That throws:
I/SqliteDatabaseCpp(8031): sqlite returned: error code = 14, msg = cannot open file at line 27701 of [8609a15dfa], db=/data/data/com.ten24.androidtest/databases/webview.db
Although - it sounds like maybe your apps can't write at all - since you can't even open a db. May be worth while posting to the Google Groups and seeing if they know something.
Few points:
- The database error seems to be always there but it still works.
- The File error is because getDirectory('xxx',{create:true}) throws error if the directory exists. getDirectory('xxx',{create:true,exclusive:false}) returns the directory if it exists.
- Also nested directory creation doesn't work. You can only create one folder at a time.
- In order to use file system make sure phone is _not_ connected in USB mode. Because connecting it in USB mode un-mounts the SD card. In DroidX the come is called "Charge Only".
I have to say, iOS development seems much more easier (trouble free) than Andriod.
1) No db errors for me.
2) When my directory existed, I never got an error.
3) Didn't test nested.
4) For me, my phone was in Charge Mode and it worked ok. It didn't work in disk mode.
Found this thread about DB error where Simon says he has always seen it and to just ignore it:
http://groups.google.com/group/phonegap/browse_thr...
I removed the offending line and tested the fix in 2.1, 2.2, 2.3 and 4.0. The fix is checked in for PhoneGap 1.4 which should be out early next week.
;)
Thanks Simon - this is great to hear!
Thanks a lot for the quick response. Is 1.4 available to play with?
You can always grab the latest code from:
https://git-wip-us.apache.org/repos/asf?p=incubato...
or the regularly updated mirror on github:
https://github.com/apache/incubator-cordova-androi...
You won't have long to wait for the official 1.4 though as it should be out on Monday.
I ran in a little problem, you compare the known files and the files downloadable files, do you have an idea how to identify a file tha needs to be updated?
What i do is i update the data in my app (download a JSON file) but only want to download the file if it has been changed since the last update.
regards
Alex
On the client side (your PhoneGap app), your code requests this server, and then compares it to the local directory.
I found a solution. I request a php page which returns an json containing a date.
I create an empty file on the phone and name it like the date.
Next time i check for the "datefile" if it exists i am not updating if it doesnt i download the update.
Thanks for pointing me in the right direction.
Completely forgot about localStorage.
If i may i have one last question (as i am not very familiar with jquery).
I use your code to download a file but it seems that the app "starts" before the file is complete.
Result, old data is shown instead of the new on. Do you have any idea how to tell the app to wait for the download to complete and display a short message?
thx for all your help
I use sencha touch for my app and the app starts before the download ist completed.
My idea is that i load the index.html and do all the update stuff. When all the updates are done (or not if the user has no internet connection) i want to move on to another page where i start the actual app.
In your example you would just download all the pictures and start the display of the pictures after all the downloading is done.
Ok, i am downloading multiple files. So what i could do is check if it is the last file and then move the user to the next page.
ft.download("http://my_download_path/"; + escape(res[x]), dlPath, function(){
if ($x == res.length){
document.loction = 'app.html';
}
}, onError);
In the onError function i put also a location change if something goes wrong during download.
And before i even start downloading i check with
if (navigator.online)....
that an internet connection exists.
Ok..thx again for your patience and your help. Hope this was the last time i had to bother you :-)
jQuery 1.5 added easier support for grouping together such tasks such that you can "do x when all is done". This feature is based on a feature called Deferreds. I'd do a bit of reading up on it. (Personally, I still struggle with the syntax.)
For my app I didn't have to worry about it.
I should force myself to update the app to use it. :)
Too bad.
Is there a way to download files NOT asynch?
Get list of files from the JSON and than download one after the other?
As far as i understand, the asynch method should prevent the app from making the user wait till all is loaded.
But when updating data i will tell the user that the app is updating and wait for the update to finish.
I'd really check the jQuery docs. I know I made it sound kind of scary, but it DOES work, and is very doable. It's just a topic I'm having a hard time wrapping my head around.
Maybe i should first try to fully understand how all those callback functions work ;-)
I'll give it a try.
PS: there is some extra code in here that you can remove, but you will get the idea.
/* download all the images */
function downloadImages(data){
var imageList = data.IMAGELIST;
var imagePath = data.IMAGEPATH;
var imageName = "";
imageData.imagePath = imagePath;
imageData.imageList = [];
var colMap = new Object();
//create column map
for (var i = 0; i < imageList.COLUMNS.length; i++) {
colMap[imageList.COLUMNS[i]] = i;
}
for (var i = 0; i < imageList.DATA.length; i++) {
imageName = imageList.DATA[i][colMap["NAME"]];
imageData.imageList.push(imageName);
}
consoleLog('Total images to download: ' + imageData.imageList.length);
downloadImage();
}
/* download image syncronously */
function downloadImage(){
if(imageData.imageList.length > 0){
var imageName = imageData.imageList[0];
var imagePath = imageData.imagePath;
consoleLog('Start file download: ' + imagePath + imageName);
//check if file exists, pass the function as new closure, so the variable is preserved
imageDir.getFile(imageName, {create: false}, successImageExists, (function(name, path){
return function(){
failImageExists(name, path)
}
})(imageName, imagePath));
}
}
/* image exists */
function successImageExists(file){
consoleLog('Image Exists: ' + file.name);
updateImageDownloadProgress();
}
/* image doesn't exists */
function failImageExists(imageName,imagePath){
consoleLog('image does not Exists');
//start download
consoleLog('Download Image To: ' + imageDir.fullPath + '/' + imageName);
ft.download(imagePath + escape(imageName), imageDir.fullPath + '/' + imageName, onImageDownload, onImageDownloadFail);
}
/* on image download fail */
function onImageDownloadFail(error){
consoleLog('image not downloaded. Error: ' + JSON.stringify(error));
updateImageDownloadProgress();
}
/* on image download success */
function onImageDownload(image){
updateImageDownloadProgress();
}
/* update image download progress */
function updateImageDownloadProgress(){
imageData.imageList = imageData.imageList.slice(1);
consoleLog('images left to download: ' + imageData.imageList.length);
if(imageData.imageList.length == 0){
imagesDownloaded = true;
} else {
downloadImage();
}
}
http://pastebin.com/kNvF166F
But... I will say this. Your implementation is not one I agree with. It looks like it would work, but you really should look at deferreds. It handles this much more elegantly.
If anything, you've convinced me to try to find time to update my demo. ;)
If i get deferred to work i will let you know and post my solution.
Sumit, thx for giving me an good idea for a workaround.
Thx to both of you for the great support :-)
I guess you set some of the variables used in your code outside.
e.g.
downloadImages(data)v <= what does Data contain
imageDir. <= where and how is this defined
imageData <= same as above
Would be great if you could show me how you modified raymonds code to work with your example.
Sorry for bothering you again :-)
Thx for your great help.
Data is a struct returned from CF.
data.imagePath = "http://domain.com/path/to/image/folder";
data.imageList = list of image names (converted to JSON from query by CF)
Imagelist comes from cfdirectory.
imageDir is the reference to image directory in FileSystem. Same as DATADIR in Ray's example.
imageData is a global JS variable to hold the image name array, as files are downloaded image names are removed from this array.
HTH,
Sumit
so data.imageList is almost the same as Raymonds
$.get("http://www.raymondcamden.com/demos/2012/jan/17/ima...;, {}, function(res) {
....
}, "json");
So what i could do is take your function downloadImages() and change it to:
function downloadImages(data){
var imagePath = 'http://path_to_my_files/';
$.get("http://path_to_my_JSON/json.php";, {}, function(res) {
var imageName = "";
imageData.imagePath = imagePath;
imageData.imageList = [];
var colMap = new Object();
//create column map
for (var i = 0; i < res.COLUMNS.length; i++) {
colMap[res.COLUMNS[i]] = i;
}
for (var i = 0; i < res.DATA.length; i++) {
imageName = res.DATA[i][colMap["NAME"]];
imageData.imageList.push(imageName);
}
consoleLog('Total images to download: ' + imageData.imageList.length);
}, "json");
downloadImage();
}
My webpage (json.php) returns something like this:
["files.json","data.json","dates.json"] (the files i want to download)
Ist this what your column mapper expects?
Btw. just to give you a short explaination why i ask so much instead of trying it all by myself.
This is my first try with HTML5, javascript and phonegap. I promised a friend to help him with a little non commercial app and i will be on holiday by the end of the week for 4 weeks ;-)
http://pastebin.com/gZQXL8P3
but it breaks after
..console.log("Start downloading...");
within downloadImages()
Btw. the download process is still triggerd by the init() script from Rays example. Is this the best way to start the download and make the page "wait" until the download is finished and then move on or relocate to another page?
i used console.Log instead of console.log....
in the code above what do you mean by : imagelister.cfc?method=listimages
I am going to use this code but only stuck at above line. i am new to phonegap and i9 need to synchronize images from remote server. please help.
Good tutorial, very helpful. However, I still got error while file downloading. I copied all code to my project, when I run it on my real mobile (android 2.3.5), the error came out. It says:
-----
03-20 23:51:34.863: E/FileTransfer(16025): Error while downloading
03-20 23:51:34.863: E/FileTransfer(16025): java.io.IOException: Error while downloading
03-20 23:51:34.863: E/FileTransfer(16025): at org.apache.cordova.FileTransfer.download(FileTransfer.java:429)
03-20 23:51:34.863: E/FileTransfer(16025): at org.apache.cordova.FileTransfer.execute(FileTransfer.java:102)
03-20 23:51:34.863: E/FileTransfer(16025): at org.apache.cordova.api.PluginManager$1.run(PluginManager.java:150)
03-20 23:51:34.863: E/FileTransfer(16025): at java.lang.Thread.run(Thread.java:1019)
-----
Any thoughts?
Checking your local cache....
I've gotten reports that 1.5 may be an issue. Sometime today/this weekend I'll try to replicate.
Is there a way to set the gotFIles() function to do a scandir type of action so it can also check all subdirectories for any matched files?
One other issue I have run into is finding a way to trigger an "all files finished downloading" event. I have tried inserting a callback function after the .get function, in the get function if the last resource has been reached, and a handful of other places. It seems that the callback function will execute after the JSON object is scanned, but before the actual download process completes. I'm looking for something similar to the FileWriter's property of readyState that can indicate a "DONE" state. I have really had a hard time finding any documentation on the File Transfer download method.
I have downloaded image in emulator and its works fine but somehow on real device it is not downloading the images. don't where it stuck... ?? any idea about that?
Thanks in advance.
Yes, please see my earlier comment.
http://www.raymondcamden.com/index.cfm/2012/1/19/D...
Sumit
However now my next problem: FileTransfer.download gives me ABORT_ERR. Any idea why? I'm trying to download a file to the TEMPORARY filesystem
<feature name="http://api.phonegap.com/1.0/file"/>;
<access origin="https://mydomain.com"; />
fixed it
[Add Comment] [Subscribe to Comments]