Uploaded image for project: 'Commons IO'
  1. Commons IO
  2. IO-175

IOUtils.doCopyFile() issues with very large files and closing file input streams

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 2.0
    • Utilities
    • None

    Description

      I've noticed the code in http://svn.apache.org/repos/asf/commons/proper/io/trunk/src/java/org/apache/commons/io/FileUtils.java for doCopyFile() contains:

      output.transferFrom(input, 0, input.size());

      I know from experience (I had code that did this previously) this will not work on large files, at least under Windows. By default, the transferFrom() method will try to create a single memory map equal to the size of the input. If you are running a 32-bit process, and are trying to copy a file gigabytes in size, this will fail as you won't have enough virtual address space to create the memory map. We had to use transferFrom() in smaller chunk sizes to work around this.

      Also I believe in the code of this method, the file input streams need to be closed explicitly, since closing the channel objects that are derived from them will not close any resources held by the file input stream objects themselves.

      Attachments

        Activity

          People

            niallp Niall Pemberton
            sits David Sitsky
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: