Hadoop Common
  1. Hadoop Common
  2. HADOOP-10364

JsonGenerator in Configuration#dumpConfiguration() is not closed

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:

      Description

          JsonGenerator dumpGenerator = dumpFactory.createJsonGenerator(out);
      

      dumpGenerator is not closed in Configuration#dumpConfiguration()

      Looking at the source code of org.codehaus.jackson.impl.WriterBasedGenerator#close(), there is more than flushing the buffer.

      1. HADOOP-10364.2.patch
        3 kB
        Sanghyun Yun
      2. HADOOP-10364.patch
        0.6 kB
        Sanghyun Yun

        Activity

        Hide
        Sanghyun Yun added a comment -

        I modify "flush()" at the end to "close()".
        close function will force flushing of output and close underlying output stream.

        Show
        Sanghyun Yun added a comment - I modify "flush()" at the end to "close()". close function will force flushing of output and close underlying output stream.
        Hide
        Tsuyoshi Ozawa added a comment -

        HI Sanghyun Yun, thanks for your contribution. To avoid a resource leak, we should add try {} finally {} block and close dumpGenerator in finally block.

        +    dumpGenerator.close();
        

        We should use IOUtils instead of using close() method directory, because close() can throw IOException and null check.

        Show
        Tsuyoshi Ozawa added a comment - HI Sanghyun Yun , thanks for your contribution. To avoid a resource leak, we should add try {} finally {} block and close dumpGenerator in finally block. + dumpGenerator.close(); We should use IOUtils instead of using close() method directory, because close() can throw IOException and null check.
        Hide
        Tsuyoshi Ozawa added a comment -

        s/directory/directly/

        Show
        Tsuyoshi Ozawa added a comment - s/directory/directly/
        Hide
        Sanghyun Yun added a comment -

        Tsuyoshi Ozawa Thanks for your review! I added try-finally block and modified code using IOUtils.cleanup() method.

        Show
        Sanghyun Yun added a comment - Tsuyoshi Ozawa Thanks for your review! I added try-finally block and modified code using IOUtils.cleanup() method.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch
        against trunk revision 258c7d0.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The test build failed in hadoop-common-project/hadoop-common

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4615//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4615//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch against trunk revision 258c7d0. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The test build failed in hadoop-common-project/hadoop-common +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4615//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4615//console This message is automatically generated.
        Hide
        Tsuyoshi Ozawa added a comment -

        It looks good to me(non-binding).

        Show
        Tsuyoshi Ozawa added a comment - It looks good to me(non-binding).
        Hide
        Sanghyun Yun added a comment -

        Thank you, Tsuyoshi Ozawa.

        Show
        Sanghyun Yun added a comment - Thank you, Tsuyoshi Ozawa .
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 39s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 30s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 4s The applied patch generated 2 new checkstyle issues (total was 176, now 176).
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 32s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 23m 9s Tests passed in hadoop-common.
            60m 8s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 6ae2a0d
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/diffcheckstylehadoop-common.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/whitespace.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 39s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 4s The applied patch generated 2 new checkstyle issues (total was 176, now 176). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 9s Tests passed in hadoop-common.     60m 8s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6ae2a0d checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6442/console This message was automatically generated.
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 27s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 28s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 7s The applied patch generated 2 new checkstyle issues (total was 176, now 176).
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 23m 5s Tests passed in hadoop-common.
            59m 56s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a60f78e
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/diffcheckstylehadoop-common.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/whitespace.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 28s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 7s The applied patch generated 2 new checkstyle issues (total was 176, now 176). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 5s Tests passed in hadoop-common.     59m 56s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a60f78e checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6577/console This message was automatically generated.
        Hide
        Haohui Mai added a comment -

        Please update the patch to use Java 7's try-with-resource statement. Thanks.

        Show
        Haohui Mai added a comment - Please update the patch to use Java 7's try-with-resource statement. Thanks.
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 53s trunk passed
        +1 compile 8m 56s trunk passed with JDK v1.8.0_66
        +1 compile 9m 20s trunk passed with JDK v1.7.0_85
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 1m 7s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        -1 findbugs 1m 58s hadoop-common-project/hadoop-common in trunk has 1 extant Findbugs warnings.
        +1 javadoc 0m 55s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 7s trunk passed with JDK v1.7.0_85
        +1 mvninstall 1m 42s the patch passed
        +1 compile 8m 39s the patch passed with JDK v1.8.0_66
        +1 javac 8m 39s the patch passed
        +1 compile 9m 9s the patch passed with JDK v1.7.0_85
        +1 javac 9m 9s the patch passed
        -1 checkstyle 0m 17s Patch generated 2 new checkstyle issues in hadoop-common-project/hadoop-common (total was 176, now 176).
        +1 mvnsite 1m 6s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 0m 57s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 6s the patch passed with JDK v1.7.0_85
        -1 unit 7m 58s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 8m 4s hadoop-common in the patch failed with JDK v1.7.0_85.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        74m 31s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.permission.TestFsPermission
          hadoop.io.compress.TestCompressorDecompressor
        JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor
        JDK v1.7.0_85 Failed junit tests hadoop.fs.permission.TestFsPermission
          hadoop.metrics2.impl.TestMetricsSystemImpl
          hadoop.io.compress.TestCompressorDecompressor
        JDK v1.7.0_85 Timed out junit tests org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch
        JIRA Issue HADOOP-10364
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 57109f825574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 077fa52
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
        JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 75MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 53s trunk passed +1 compile 8m 56s trunk passed with JDK v1.8.0_66 +1 compile 9m 20s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 16s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 58s hadoop-common-project/hadoop-common in trunk has 1 extant Findbugs warnings. +1 javadoc 0m 55s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 7s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 42s the patch passed +1 compile 8m 39s the patch passed with JDK v1.8.0_66 +1 javac 8m 39s the patch passed +1 compile 9m 9s the patch passed with JDK v1.7.0_85 +1 javac 9m 9s the patch passed -1 checkstyle 0m 17s Patch generated 2 new checkstyle issues in hadoop-common-project/hadoop-common (total was 176, now 176). +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 57s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 6s the patch passed with JDK v1.7.0_85 -1 unit 7m 58s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 8m 4s hadoop-common in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 74m 31s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.permission.TestFsPermission   hadoop.io.compress.TestCompressorDecompressor JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor JDK v1.7.0_85 Failed junit tests hadoop.fs.permission.TestFsPermission   hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.io.compress.TestCompressorDecompressor JDK v1.7.0_85 Timed out junit tests org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12665692/HADOOP-10364.2.patch JIRA Issue HADOOP-10364 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 57109f825574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 077fa52 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8126/console This message was automatically generated.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development