Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10383

Safely close resources in DFSTestUtil

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • test
    • None
    • Reviewed

    Description

      There are a few of methods in DFSTestUtil that do not close the resource safely, or elegantly. We can use the try-with-resource statement to address this problem.

      Specially, as DFSTestUtil is popularly used in test, we need to preserve any exceptions thrown during the processing of the resource while still guaranteeing it's closed finally. Take for example,the current implementation of DFSTestUtil#createFile() closes the FSDataOutputStream in the finally block, and when closing if the internal DFSOutputStream#close() throws any exception, which it often does, the exception thrown during the processing will be lost. See this test failure, and we have to guess what was the root cause.

      Using try-with-resource, we can close the resources safely, and the exceptions thrown both in processing and closing will be available (closing exception will be suppressed). Besides the try-with-resource, if a stream is not necessary, don't create/close it.

      Attachments

        1. HDFS-10383-branch-2.000.patch
          15 kB
          Mingliang Liu
        2. HDFS-10383.003.patch
          17 kB
          Mingliang Liu
        3. HDFS-10383.002.patch
          15 kB
          Mingliang Liu
        4. HDFS-10383.001.patch
          16 kB
          Mingliang Liu
        5. HDFS-10383.000.patch
          9 kB
          Mingliang Liu

        Activity

          People

            liuml07 Mingliang Liu
            liuml07 Mingliang Liu
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: