Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13523

Fix connection leak in ORC RecordReader and refactor for unit testing

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.1, 2.1.0
    • Component/s: ORC
    • Labels:
      None

      Description

      In RecordReaderImpl, a MetadataReaderImpl object was being created (opening a file), but never closed, causing a leak. This change closes the Metadata object in RecordReaderImpl, and does substantial refactoring to make RecordReaderImpl testable:

      • Created DataReaderFactory and MetadataReaderFactory (plus default implementations) so that the create() methods can be mocked to verify that the objects are actually closed in RecordReaderImpl.close()
      • Created MetadataReaderProperties and DataReaderProperties to clean up argument lists, making code more readable
      • Created a builder() for RecordReaderImpl to make the code more readable
      • DataReader and MetadataReader now extend closeable (there was no reason for them not to in the first place) so I can use the guava Closer interface: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/Closer.html
      • Use the Closer interface to guarantee that regardless of if either close() call fails, both will be attempted (preventing further potential leaks)
      • Create builders for MetadataReaderProperties, DataReaderProperties, and RecordReaderImpl to help with code readability

        Attachments

        1. HIVE-13523.patch
          42 kB
          Thomas Poepping

          Activity

            People

            • Assignee:
              poeppt Thomas Poepping
              Reporter:
              poeppt Thomas Poepping
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: