Hive
  1. Hive
  2. HIVE-1924

checkformat implementations leak handles

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      In validateInput, Reader constructors of SequenceFile and RCFile throw exceptions to indicate that the format is incorrect, but the close is not called in a finally block.

        Activity

        Krishna Kumar created issue -
        Hide
        Krishna Kumar added a comment -

        Not sure how to test this automatically, since the readers are transient. The way I actually tested is by

        • running on a nfs mounted directory (since nfs creates .nfsxxxxxx files for open files which have been deleted)
        • pausing the code after a load command (with conf hive. is executed
        • using lsof to list open files by the process
        Show
        Krishna Kumar added a comment - Not sure how to test this automatically, since the readers are transient. The way I actually tested is by running on a nfs mounted directory (since nfs creates .nfsxxxxxx files for open files which have been deleted) pausing the code after a load command (with conf hive. is executed using lsof to list open files by the process
        Krishna Kumar made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Krishna Kumar added a comment -

        Tested only manually with hive.fileformat.check set to true

        Show
        Krishna Kumar added a comment - Tested only manually with hive.fileformat.check set to true
        Krishna Kumar made changes -
        Attachment HIVE.1924.patch.txt [ 12469124 ]
        Hide
        Krishna Kumar added a comment -

        Hmm, The patch is not correct. Setting hive.checkformat=false does seem to stop the leaks though. Investigating...

        Show
        Krishna Kumar added a comment - Hmm, The patch is not correct. Setting hive.checkformat=false does seem to stop the leaks though. Investigating...
        Krishna Kumar made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Krishna Kumar added a comment -

        Problem referenced in HADOOP-5476 and HIVE-1185

        Show
        Krishna Kumar added a comment - Problem referenced in HADOOP-5476 and HIVE-1185
        Krishna Kumar made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Duplicate [ 3 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Krishna Kumar
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development