Uploaded image for project: 'Crunch (Retired)'
  1. Crunch (Retired)
  2. CRUNCH-53

AvroFileReaderFactory does not close input files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.3.0
    • IO
    • None

    Description

      The AvroFileReaderFactory read() method does not close its DataFileReader. With the Hadoop NativeS3FileSystem this can lead to the following warning:

      org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream: Successfully released HttpMethod in finalize(). You were lucky this time... Please ensure S3 response data streams are always fully consumed or closed.
      WARN [2012-08-28 19:26:16,035] org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream: Attempting to release HttpMethod in finalize() as its response data stream has gone out of scope. This attempt will not always succeed and cannot be relied upon! Please ensure S3 response data streams are always fully consumed or closed to avoid HTTP connection starvation.

      Attachments

        1. CRUNCH-53-autoclose.patch
          8 kB
          Shawn Smith

        Activity

          People

            Unassigned Unassigned
            ssmith Shawn Smith
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: