Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10383

Safely close resources in DFSTestUtil

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There are a few of methods in DFSTestUtil that do not close the resource safely, or elegantly. We can use the try-with-resource statement to address this problem.

      Specially, as DFSTestUtil is popularly used in test, we need to preserve any exceptions thrown during the processing of the resource while still guaranteeing it's closed finally. Take for example,the current implementation of DFSTestUtil#createFile() closes the FSDataOutputStream in the finally block, and when closing if the internal DFSOutputStream#close() throws any exception, which it often does, the exception thrown during the processing will be lost. See this test failure, and we have to guess what was the root cause.

      Using try-with-resource, we can close the resources safely, and the exceptions thrown both in processing and closing will be available (closing exception will be suppressed). Besides the try-with-resource, if a stream is not necessary, don't create/close it.

        Attachments

        1. HDFS-10383.000.patch
          9 kB
          Mingliang Liu
        2. HDFS-10383.001.patch
          16 kB
          Mingliang Liu
        3. HDFS-10383.002.patch
          15 kB
          Mingliang Liu
        4. HDFS-10383.003.patch
          17 kB
          Mingliang Liu
        5. HDFS-10383-branch-2.000.patch
          15 kB
          Mingliang Liu

          Activity

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              liuml07 Mingliang Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: