Suggestions for Database Syncing with PhoneGap/Cordova

This post is more than 2 years old.

Shai asked me an interesting question that I thought would be good to discuss here, and get feedback from my audience. Their question was:

Hello Ray,

I'm a long follower of your blog. I am also a MobileFirst & PhoneGap developer.

I wanted to consult with you, if I may, what is your suggestion about developing a PhoneGap app (not MobileFirst) which has to be synched with a remote DB. The user populated data on the app, and if the app is offline than it is stored locally. When the network returns, the app must upload the data to the remote server and it has to be done seamlessly. The amount of data in terms of records and/or size could become quite "big". I don't suppose it could 5 MB but we have to provide a solution in case it does, as part of it is pictures taken by the user.

Any suggestion would be most appreciated. Do you have a "reliable" plug-in you could recommend?

Thanks and BR's,

Shai.

I have no idea what BRs are, but you are welcome. ;) So first off, this can definitely be done, "by hand", if you would like. You can use the Network plugin to detect when a user is on or offline. You can use regular old Ajax to send the data, being sure to only send what you need to send, i.e. data that is new. Storing data locally you've got a few options as well. WebSQL is baked in, but has limits. You can also use the SQLite plugin. I haven't used this yet myself but plan on playing with it soon. As far as I know though it follows the same concept as WebSQL. Back in 2012, I blogged on this topic myself, Adding database synchronization to your PhoneGap project. (Hard to believe I've been blogging about PhoneGap/Cordova that long.)

How about libraries? As far as I know, Parse's offline support is only for Android now. I recently played with PouchDB (Connecting PouchDB to Cloudant on IBM Bluemix) and I really, really like it, but haven't used it in production yet. Firebase has offline support too, but I haven't yet tried their product.

All of these solutions assume that you are ok with using their own database of course and not your own. If it must be your database than you'll want to go with the manual approach I described above.

Raymond Camden's Picture

About Raymond Camden

Raymond is a senior developer evangelist for Adobe. He focuses on document services, JavaScript, and enterprise cat demos. If you like this article, please consider visiting my Amazon Wishlist or donating via PayPal to show your support. You can even buy me a coffee!

Lafayette, LA https://www.raymondcamden.com

Archived Comments

Comment 1 by ⊥ᵒᵚ Cᵸᵎᶺᵋ⤷╨ᵒᵘ posted on 5/15/2015 at 1:17 PM

Have you seen https://github.com/orbitalo... ? Seems to do the job.

Comment 2 by chandramuralis posted on 5/15/2015 at 5:03 PM

BR - Best Regards

Comment 3 (In reply to #1) by Raymond Camden posted on 5/15/2015 at 5:48 PM

Nope, I had not, which is exactly why I posted this. Thanks!

Comment 4 by Chris Geirman posted on 5/15/2015 at 8:07 PM

Ray, when using something like PouchDB or Firebase, is there any limit to the size of data saved on the client? Another somewhat loosely related question (stick with me, I'll bring it around)... this user wants to save "pictures" which I assume means they want to use the camera to take a picture and save it. These photos can themselves be 5MB. In any case, is it possible to... save the photo locally offline, sync it online once reconnected, and store it in a custom album on the user's device? The first two parts are related to this post and is probably possible within certain size limits assuming we save the images as Base64, but the second part with regard to creating custom album and saving the image to that album (rather than to their generic gallery). Is that possible with PhoneGap/Cordova or does that now start leaning more toward necessitating a native solution?

Comment 5 (In reply to #1) by Chris Geirman posted on 5/15/2015 at 8:09 PM

Thanks +1

Comment 6 (In reply to #4) by Gary F posted on 5/17/2015 at 12:29 AM

Chris Geirman , Instead if wasting space in a local db to store photos I would suggest storing the jpeg's file path & name in the local database. Then when the network is back online the app would upload the photo as a jpeg from the file system. This will save a huge amount of space in the local database and possible duplication (a copy on the file system and a copy in the db).

Comment 7 (In reply to #6) by Chris Geirman posted on 5/17/2015 at 5:47 AM

Gary F, thanks for your response. I wasn't sure if I'd have access to the physical file path. As I understand it (please correct me if I'm wrong), the camera will hold a pointer to the place in memory where the photo is stored (not the file path), but once it releases that pointer I'd no longer have a way to access the file system asset from the app for security reasons. I've not yet done this myself, so I just took that as gospel. Again, please feel free to educate me. Also, do you think it's possible to create a special named album for my app? I've not been able to find that answer yet anywhere.

Comment 8 (In reply to #7) by Raymond Camden posted on 5/17/2015 at 1:43 PM

When you take a *new* pic, you can use an option, "saveToPhotoAlbum", to have the picture saved, but I don't think the FileURI you get *initially* will probably be a temp location. You could flip it and tell it NOT to save, and then YOU copy manually. That way you'd have access to the final url.

Comment 9 (In reply to #4) by Nolan Lawson posted on 5/19/2015 at 2:22 AM

PouchDB's data limits are pretty generous: basically 50MB in iOS and unlimited on Android. Also you can get unlimited space on iOS by adding the SQLite Plugin: http://pouchdb.com/faq.html...

For storing images, PouchDB also has an "attachment" API, which stores the data as binary blobs if possible, so it's quite efficient. And yes, those attachments sync back to CouchDB. :)

Although as others have said, it might be a shame to have images stored twice - once in PouchDB and once on the filesystem. And the current replicator algorithm uploads the images as base64-encoded strings, which may be unmanageable for 5MB images. (Although that should change soon; we're working on multipart/mixed support.) So there are some downsides, but overall it works. :)

Comment 10 (In reply to #9) by Raymond Camden posted on 5/19/2015 at 12:40 PM

Thanks for the details, Nolan.

Comment 11 by Victor Mendoza posted on 1/14/2016 at 4:30 PM

I am aware of Hoodie (http://hood.ie) seems interesting. Haven't tested it yet, honestly I haven't found too much info about , it seems it is not production ready but maybe it is check worthy. I think I would use PouchDB, the documentation is quite complete and ease to follow.

On the other hand Firebase does offer true offline support but sadly not for JavaScript yet. I have and cordova-ionic app working with Firebase and was willing to have true offline support as they said they were working towards it but in the latest post in their forums they have said that there are no clear plans to develop offline support for JS that why I am looking for other true offline support options for a new app. Do not misunderstand me, Firebase is great but for offline support with JS you will need to implement a custom solution or look for other options out there.

Comment 12 by Domingo posted on 8/9/2017 at 10:50 PM

Hello, I just started working with Pouchdb on Phonegap and have had a tough time using the tools. there doens't seem to be good documentation or troubleshooting guides for the PouchDB product. Are there any forums where one can ask questions from the CouchDB/PouchDB team. Right now I have an issue that PouchDB doens't sync with CouchDB on the android/phonegap but it does when running on the chrome browser.
Any help is appreciated.