Uploaded image for project: 'Commons IO'
  1. Commons IO
  2. IO-251

Add new read method "toByteArray" to handle InputStream with known size

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 2.1
    • Component/s: Utilities
    • Labels:
      None

      Description

      I suggest adding a new method toByteArray(InputStream input, int size) to IoUtils to handle known size InputStream (e.g. files).
      Current implementation based on copy method consumes more memory than necessary (at least the double) to transform a InputStream into a byte array.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/commons-io/pull/38

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/commons-io/pull/38
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user ilmarmors opened a pull request:

          https://github.com/apache/commons-io/pull/38

          FileUtils.readFileToByteArray - optimize reading of files with known size

          IO-251 added optimization for reading files with known size as byte arrays, but it was reverted in IO-453 due to regression, because File.length() method may return 0 for path files denoting system-dependent entities such as devices or pipes

          More optimal solution is to treat 0 as unknown size, but if size is known (> 0) use more efficient approach and read content into pre-allocated byte array with exact size.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/ilmarmors/commons-io master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/commons-io/pull/38.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #38


          commit 3ce13884f3b2d78c6ca6cbf9a85c240c6b6532c3
          Author: Ilmars Poikans <ilmars@delibero.lv>
          Date: 2017-07-02T13:01:23Z

          FileUtils.readFileToByteArray - optimize reading of files with known size


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user ilmarmors opened a pull request: https://github.com/apache/commons-io/pull/38 FileUtils.readFileToByteArray - optimize reading of files with known size IO-251 added optimization for reading files with known size as byte arrays, but it was reverted in IO-453 due to regression, because File.length() method may return 0 for path files denoting system-dependent entities such as devices or pipes More optimal solution is to treat 0 as unknown size, but if size is known (> 0) use more efficient approach and read content into pre-allocated byte array with exact size. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ilmarmors/commons-io master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/commons-io/pull/38.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #38 commit 3ce13884f3b2d78c6ca6cbf9a85c240c6b6532c3 Author: Ilmars Poikans <ilmars@delibero.lv> Date: 2017-07-02T13:01:23Z FileUtils.readFileToByteArray - optimize reading of files with known size
          Hide
          sebb@apache.org Sebb added a comment -

          The change to readFileToByteArray caused a regression, see IO-453

          Show
          sebb@apache.org Sebb added a comment - The change to readFileToByteArray caused a regression, see IO-453
          Hide
          garydgregory Gary Gregory added a comment -

          Closing, we released version 2.1.

          Show
          garydgregory Gary Gregory added a comment - Closing, we released version 2.1.
          Hide
          sebb@apache.org Sebb added a comment -

          Thanks, added to SVN

          Show
          sebb@apache.org Sebb added a comment - Thanks, added to SVN
          Hide
          malbini Marco Albini added a comment -

          Added patch proposal file with:

          • New IOUtils method byte[] toByteArray(InputStream input, int size)
          • New IOUtils method byte[] toByteArray(InputStream input, long size)
          • Modified FileUtils method byte[] readFileToByteArray(File file)
          • Unit Test
          Show
          malbini Marco Albini added a comment - Added patch proposal file with: New IOUtils method byte[] toByteArray(InputStream input, int size) New IOUtils method byte[] toByteArray(InputStream input, long size) Modified FileUtils method byte[] readFileToByteArray(File file) Unit Test

            People

            • Assignee:
              Unassigned
              Reporter:
              malbini Marco Albini
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development