Apache Cordova
  1. Apache Cordova
  2. CB-2537

Out of memory errors : FileTransfer download fails for large files over a certain size

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.7.0
    • Component/s: iOS
    • Labels:
      None
    • Environment:

      Mac OSX 10.7 / Xcode 4.6 / iOS FW 5 & 6

      Description

      Hello,
      on iOS (FW 5 & 6), i'm facing a problem with the download function of FileTransfer API.
      Downloading files over a certain size (approx 300MB / 400MB) leads to a out of memory error.
      is there any way to bypass this limit ? by applying a chunked mode like in the upload function ?

      Thanks for your help

        Issue Links

          Activity

          Show
          Andrew Grieve added a comment - Committed. https://git-wip-us.apache.org/repos/asf?p=cordova-ios.git;a=commit;h=d6a35768e74bda0e77a1bbe0aba8fcfe41c7b5c0
          Hide
          Shazron Abdullah added a comment -

          Great! Looking forward to it in 2.7.0

          Show
          Shazron Abdullah added a comment - Great! Looking forward to it in 2.7.0
          Hide
          Ian Clelland added a comment -
          Show
          Ian Clelland added a comment - Pull request sent: https://github.com/apache/cordova-ios/pull/39
          Hide
          Ian Clelland added a comment -

          I've implemented streaming downloads on my fork (for successful downloads only; HTTP errors are still handled by accumulating the response in memory, and then returning it in the error callback)

          Profiling this on an 800MB download, I watched the memory usage climb during the download (using the existing code). With this patch, usage stays flat; each block is written to the filesystem as it is received.

          Show
          Ian Clelland added a comment - I've implemented streaming downloads on my fork (for successful downloads only; HTTP errors are still handled by accumulating the response in memory, and then returning it in the error callback) Profiling this on an 800MB download, I watched the memory usage climb during the download (using the existing code). With this patch, usage stays flat; each block is written to the filesystem as it is received.
          Hide
          Shazron Abdullah added a comment -

          This shouldn't be hard to write. I had an older plugin that did chunked downloads: https://github.com/filmaj/Hydra/tree/master/ext/iOS/plugins/BinaryDownloaderPlugin

          Show
          Shazron Abdullah added a comment - This shouldn't be hard to write. I had an older plugin that did chunked downloads: https://github.com/filmaj/Hydra/tree/master/ext/iOS/plugins/BinaryDownloaderPlugin
          Hide
          charbon bleu added a comment -

          I'm trying to create a plugin which will be the same as the fileTransfer Cordova plugin but changing only the store and writing method but I'm a new bee on ios development.
          Thank you for your help

          Show
          charbon bleu added a comment - I'm trying to create a plugin which will be the same as the fileTransfer Cordova plugin but changing only the store and writing method but I'm a new bee on ios development. Thank you for your help
          Hide
          Andrew Grieve added a comment -

          Yep, that's exactly what we should be doing. The place to make the change is in CordovaLib/Classes/CDVFileTransfer.m. I don't expect that I'll get to this very soon, so feel free to take a stab at it

          Show
          Andrew Grieve added a comment - Yep, that's exactly what we should be doing. The place to make the change is in CordovaLib/Classes/CDVFileTransfer.m. I don't expect that I'll get to this very soon, so feel free to take a stab at it
          Hide
          charbon bleu added a comment - - edited

          I'm not and expert with iOS coding, but I found in several forum that the way the download function stores data is not the best,
          because it append the whole received data to a NSMutableData object.
          instead of that, the solution to bypass this problem is to store with the didReceiveData function each chunk of data as received in the file directly with NSFileHandle:writeData function.
          is there a way to fix this quickly with a patch or plugin correction or something like this ?

          Show
          charbon bleu added a comment - - edited I'm not and expert with iOS coding, but I found in several forum that the way the download function stores data is not the best, because it append the whole received data to a NSMutableData object. instead of that, the solution to bypass this problem is to store with the didReceiveData function each chunk of data as received in the file directly with NSFileHandle:writeData function. is there a way to fix this quickly with a patch or plugin correction or something like this ?
          Hide
          Andrew Grieve added a comment -

          Yeah, the plugin is currently accumulating the entire response in an NSMutableData and writing it to disk only after the transfer completes... Not the best idea.

          Show
          Andrew Grieve added a comment - Yeah, the plugin is currently accumulating the entire response in an NSMutableData and writing it to disk only after the transfer completes... Not the best idea.
          Hide
          Filip Maj added a comment -

          Assigning to Shaz, this is less about JavaScript and more about the platform.

          Show
          Filip Maj added a comment - Assigning to Shaz, this is less about JavaScript and more about the platform.

            People

            • Assignee:
              Ian Clelland
              Reporter:
              charbon bleu
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development