Commons IO
  1. Commons IO
  2. IO-266

FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Windows 2003 Server 64-bit

      Description

      java.io.IOException: Insufficient system resources exist to complete the requested service
      at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
      at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
      at sun.nio.ch.IOUtil.write(Unknown Source)
      at sun.nio.ch.FileChannelImpl.write(Unknown Source)
      at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
      at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
      at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
      at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
      at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)

      NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
      Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).

      Here is a test program to reproduce the issue:

      package org.test.igor;

      import java.io.File;
      import java.io.FileInputStream;
      import java.io.FileOutputStream;
      import java.io.IOException;
      import java.nio.channels.FileChannel;

      import org.apache.commons.io.FileUtils;
      import org.apache.commons.io.IOUtils;

      public class TestFileUtils {

      public static void main(String[] args){

      File src = new File("D:\\2011.1-dev\\test
      test");
      File dest = new File("\\\\ismerek1
      Shared");

      String filename = "jdk-6u19-windows-x64.exe";

      File file = new File(src, filename);
      File toFile = new File(dest, filename);

      try

      { FileUtils.copyFile(file, toFile, true); System.out.println("Successful copy"); }

      catch (IOException e1)

      { e1.printStackTrace(); }

      }
      }

        Activity

        Igor Smereka created issue -
        Igor Smereka made changes -
        Field Original Value New Value
        Affects Version/s 2.0.1 [ 12315445 ]
        Environment Windows 2003 Server 64-bit
        Description
        java.io.IOException: Insufficient system resources exist to complete the requested service
        at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
        at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
        at sun.nio.ch.IOUtil.write(Unknown Source)
        at sun.nio.ch.FileChannelImpl.write(Unknown Source)
        at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
        at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
        at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
        at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)

        NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
        Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).

        Here is a test program to reproduce the issue:

        package org.test.igor;

        import java.io.File;
        import java.io.FileInputStream;
        import java.io.FileOutputStream;
        import java.io.IOException;
        import java.nio.channels.FileChannel;

        import org.apache.commons.io.FileUtils;
        import org.apache.commons.io.IOUtils;

        public class TestFileUtils {

            public static void main(String[] args){
                
                
                File src = new File("D:\\2011.1-dev\\test\\test");
                File dest = new File("\\\\ismerek1\\Shared");
                
                String filename = "jdk-6u19-windows-x64.exe";
                
                File file = new File(src, filename);
                File toFile = new File(dest, filename);
                
                try {
                    FileUtils.copyFile(file, toFile, true);
                    System.out.println("Successful copy");
                }
                catch (IOException e1) {
                    e1.printStackTrace();
                }
            }
        }






        java.io.IOException: Insufficient system resources exist to complete the requested service
        at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
        at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
        at sun.nio.ch.IOUtil.write(Unknown Source)
        at sun.nio.ch.FileChannelImpl.write(Unknown Source)
        at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
        at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
        at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
        at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)

        NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
        Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).

        Here is a test program to reproduce the issue:

        package org.test.igor;

        import java.io.File;
        import java.io.FileInputStream;
        import java.io.FileOutputStream;
        import java.io.IOException;
        import java.nio.channels.FileChannel;

        import org.apache.commons.io.FileUtils;
        import org.apache.commons.io.IOUtils;

        public class TestFileUtils {

            public static void main(String[] args){
                
                
                File src = new File("D:\\2011.1-dev\\test\\test");
                File dest = new File("\\\\ismerek1\\Shared");
                
                String filename = "jdk-6u19-windows-x64.exe";
                
                File file = new File(src, filename);
                File toFile = new File(dest, filename);
                
                try {
                    FileUtils.copyFile(file, toFile, true);
                    System.out.println("Successful copy");
                }
                catch (IOException e1) {
                    e1.printStackTrace();
                }
            }
        }






        Arnaud BRAND made changes -
        Attachment FileUtils.java.patch [ 12478600 ]
        Sebb made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 2.1 [ 12316027 ]
        Resolution Fixed [ 1 ]
        Gary Gregory made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Igor Smereka
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development