Apache Cordova
  1. Apache Cordova
  2. CB-363

SQLite database does not grow on Android

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: 1.3.0, 1.4.0, 1.5.0
    • Fix Version/s: 1.7.0
    • Component/s: Android
    • Environment:

      Android Simulator,
      Android 2.2+
      PhoneGap (any version)

      Description

      Description
      ------------------
      If I create a database with the PhoneGap API (1.3.0 or higher) I can set an initial size. If I insert data, the size of the database file should grow if there is not enough space for all the inserted data. This is not working properly on Android, and I will get the following error:

      error.code: 4
      error.message: there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space

      When I close the app (kill the process) and reopen it, the database file is grown by ~1MB and I can insert data up to that size.

      I tried inserting all of the data in one single transaction, or with every single insert being one transaction, the result is the same. Changing the database size after creation has no effect.

      Steps to Reproduce
      ------------------
      Create a Database with the phonegap API using

      var db = window.openDatabase("test", "1.0", "Test DB", 200000);

      200000 being the size of the database, as used in the (former) API examples.

      Expected Results
      ------------------
      The database is created and I can read from and write to.
      The size of the database file should grow, so all the data can get inserted.

      Actual Results
      ------------------
      The database is created and I can read from and write to.
      When I insert data, the initial size seems to be the upper limit for the size of the db (filesize of the .db file)
      When this size is reached and I try to insert data, phonegap aborts with the error described above.

      This only happens on Android, device or simulator
      For iOS the database seems to grow "on the fly"

      1. dbgrow.zip
        684 kB
        Urs Zimmermann

        Activity

        Hide
        Walter Nicholls added a comment -

        A month later I run into this issue! And naturally I object to the 'closed won't fix' idea.

        Anyway. I hit this limit on Android 4.0. It did not occur on Chrome, or Android 3. In fact, the PhoneGap app is the only one failing.

        In this case I set the ESTIMATED SIZE (not hard limit size..) in window.openDatabase(...) call to 100000 (100k)

        Runnning in the Android browser (not cordova app) on Android 4.0, I successfully get a database of 259kb ie it clearly grew ok.
        Running packaged as a cordova app on the same machine, the application crashes with "there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space" ... needless to say, the user was never ASKED let along declined. The database file /data/data/..../app_database/file_0/00000000000001.db is 88kB, which is most likely about one transaction short of the 100k limit.

        Running in the android browser on a Samsung Galaxy Tab (Android 3), Running as a cordova app on the Galaxy - in both cases the DB file has clearly grown (without user intervention). I can't tell you file sizes as above without rooting the device, but perhaps it should tell you enough that I haven't.

        However I changed the limit in the openDatabase(..) call to 10MB, and ran app again - database grew to 260k. Then set it back down to 100k and ran again, it continued to work, probably because the space has now been allocated. Maybe if there is a way to shrink the database.

        One thing though "Is there a way to re-initialized the database programmatically without losing data" .. um, is there a way to do it with losing data? With the exception of somehow clearing the tablet memory, that is. There is no way in the Web SQL API to deleting a database: best you can do is drop and recreate the tables.

        I've had a burrow in the Cordova source code (ie DroidGap.java/Storage.java) but I can't find any inking that the size parameter is even used, let alone it being possible to resize. This suggests to me that the Webview's native implementation is being used instead. I expect what is going on is that the webview is raising some event to ask the user about extending the database size (as per the specification), but that Cordova is either blocking it, or is not implementing it when it should.

        As far as Urs's original problem (and mine) goes, the main issue we are left with is potentially setting a ridiculously high estimated database size and hoping we never are unlucky enough to hit it.

        Show
        Walter Nicholls added a comment - A month later I run into this issue! And naturally I object to the 'closed won't fix' idea. Anyway. I hit this limit on Android 4.0. It did not occur on Chrome, or Android 3. In fact, the PhoneGap app is the only one failing. In this case I set the ESTIMATED SIZE (not hard limit size..) in window.openDatabase(...) call to 100000 (100k) Runnning in the Android browser ( not cordova app) on Android 4.0, I successfully get a database of 259kb ie it clearly grew ok. Running packaged as a cordova app on the same machine, the application crashes with "there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space" ... needless to say, the user was never ASKED let along declined. The database file /data/data/..../app_database/file_0/00000000000001.db is 88kB, which is most likely about one transaction short of the 100k limit. Running in the android browser on a Samsung Galaxy Tab (Android 3), Running as a cordova app on the Galaxy - in both cases the DB file has clearly grown (without user intervention). I can't tell you file sizes as above without rooting the device, but perhaps it should tell you enough that I haven't . However I changed the limit in the openDatabase(..) call to 10MB, and ran app again - database grew to 260k. Then set it back down to 100k and ran again, it continued to work, probably because the space has now been allocated. Maybe if there is a way to shrink the database. One thing though "Is there a way to re-initialized the database programmatically without losing data" .. um, is there a way to do it with losing data? With the exception of somehow clearing the tablet memory, that is. There is no way in the Web SQL API to deleting a database: best you can do is drop and recreate the tables. I've had a burrow in the Cordova source code (ie DroidGap.java/Storage.java) but I can't find any inking that the size parameter is even used, let alone it being possible to resize. This suggests to me that the Webview's native implementation is being used instead. I expect what is going on is that the webview is raising some event to ask the user about extending the database size (as per the specification), but that Cordova is either blocking it, or is not implementing it when it should. As far as Urs's original problem (and mine) goes, the main issue we are left with is potentially setting a ridiculously high estimated database size and hoping we never are unlucky enough to hit it.
        Hide
        Urs Zimmermann added a comment -

        Thank you Joe for your work.
        By "re-initialize the database" do you mean restarting the app? Is there a way to re-initialize the database programmatically without loosing data?

        Show
        Urs Zimmermann added a comment - Thank you Joe for your work. By "re-initialize the database" do you mean restarting the app? Is there a way to re-initialize the database programmatically without loosing data?
        Hide
        Joe Bowser added a comment -

        The Android API won't let me increment a quota past what the user defines. The user has to re-initalize their database if they run into this error.

        Show
        Joe Bowser added a comment - The Android API won't let me increment a quota past what the user defines. The user has to re-initalize their database if they run into this error.
        Hide
        Joe Bowser added a comment -

        I tried to fix this by forcing the database to be grown, but this has to be done explicitly. I can't do anything on the back-end to change this, unfortunately.

        Show
        Joe Bowser added a comment - I tried to fix this by forcing the database to be grown, but this has to be done explicitly. I can't do anything on the back-end to change this, unfortunately.
        Hide
        Urs Zimmermann added a comment -

        The database file is growing already: If you kill and restart the app, then the file grows up to 5mb. On iOs the file grows without a restart.

        How would you suggest to solve this: My app is in the stores using the 200000 bytes database, which was fine for the time. For an update I need to insert more data.

        Show
        Urs Zimmermann added a comment - The database file is growing already: If you kill and restart the app, then the file grows up to 5mb. On iOs the file grows without a restart. How would you suggest to solve this: My app is in the stores using the 200000 bytes database, which was fine for the time. For an update I need to insert more data.
        Hide
        Joe Bowser added a comment -

        OK, I can recreate the behaviour, but I don't know if this is a bug per-se. The fact is that you created a database of 200000 bytes, and then you try to re-initialize the database to be larger, which fails. You then try to fill the database, and this fails because you didn't give it enough memory. I don't know if I'm comfortable with having a ballooning SQLite database, especially since the W3C spec says that you should prompt for when you reach the end.

        Moving to a future release.

        Show
        Joe Bowser added a comment - OK, I can recreate the behaviour, but I don't know if this is a bug per-se. The fact is that you created a database of 200000 bytes, and then you try to re-initialize the database to be larger, which fails. You then try to fill the database, and this fails because you didn't give it enough memory. I don't know if I'm comfortable with having a ballooning SQLite database, especially since the W3C spec says that you should prompt for when you reach the end. Moving to a future release.
        Hide
        Urs Zimmermann added a comment -

        Sample code:

        • Creates database
        • Fills the database with test data
          => Fails at row 58 with "error 4: there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space"

        Close and Restart does resolve the problem.

        Show
        Urs Zimmermann added a comment - Sample code: Creates database Fills the database with test data => Fails at row 58 with "error 4: there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space" Close and Restart does resolve the problem.
        Hide
        Urs Zimmermann added a comment -

        If you create a database of some initial size, then insert data around as much as the database size you get the above error.
        I don't know if this is according to the specification. In my case it renders the storage API useless for my app since I created a small database with an older version and I need to insert more data in the new version.

        Will provide sample code later today.

        Show
        Urs Zimmermann added a comment - If you create a database of some initial size, then insert data around as much as the database size you get the above error. I don't know if this is according to the specification. In my case it renders the storage API useless for my app since I created a small database with an older version and I need to insert more data in the new version. Will provide sample code later today.
        Hide
        Joe Bowser added a comment -

        Can you please provide some code we can use to reproduce this error, otherwise I'm going to have to close this as "Won't Fix"

        Show
        Joe Bowser added a comment - Can you please provide some code we can use to reproduce this error, otherwise I'm going to have to close this as "Won't Fix"
        Hide
        Simon MacDonald added a comment -

        This doesn't appear to be a bug to me. The Android implementation is working the way it is described in the specification:

        http://www.w3.org/TR/webdatabase/

        Show
        Simon MacDonald added a comment - This doesn't appear to be a bug to me. The Android implementation is working the way it is described in the specification: http://www.w3.org/TR/webdatabase/

          People

          • Assignee:
            Joe Bowser
            Reporter:
            Urs Zimmermann
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development