Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-5981

PBImageXmlWriter generates malformed XML

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      PBImageXmlWriter outputs malformed XML file because it closes the SnapshotDiffSection, NameSection and INodeReferenceSection incorrectly.

      1. HDFS-5981.003.patch
        9 kB
        Haohui Mai
      2. HDFS-5981.002.patch
        7 kB
        Haohui Mai
      3. HDFS-5981.001.patch
        1 kB
        Haohui Mai
      4. HDFS-5981.000.patch
        0.8 kB
        Haohui Mai

        Activity

        Haohui Mai created issue -
        Haohui Mai made changes -
        Field Original Value New Value
        Attachment HDFS-5981.000.patch [ 12629978 ]
        Haohui Mai made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Akira AJISAKA added a comment -

        Looks good.

        INodeReferenceSection and NameSection are not closed correctly.
        Would you fix them in addition?

        Show
        Akira AJISAKA added a comment - Looks good. INodeReferenceSection and NameSection are not closed correctly. Would you fix them in addition?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12629978/HDFS-5981.000.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6187//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6187//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12629978/HDFS-5981.000.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6187//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6187//console This message is automatically generated.
        Haohui Mai made changes -
        Attachment HDFS-5981.001.patch [ 12630112 ]
        Haohui Mai made changes -
        Summary PBImageXmlWriter closes SnapshotDiffSection incorrectly. PBImageXmlWriter generates malformed XML
        Haohui Mai made changes -
        Description {{PBImageXmlWriter}} outputs malformed XML file because it closes the tag {{SnapshotDiffSection}} incorrectly. {{PBImageXmlWriter}} outputs malformed XML file because it closes the {{SnapshotDiffSection}}, {{NameSection}} and {{INodeReferenceSection}} incorrectly.
        Hide
        Akira AJISAKA added a comment -

        Thank you for updating the patch! LGTM, +1.

        Show
        Akira AJISAKA added a comment - Thank you for updating the patch! LGTM, +1.
        Akira AJISAKA made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Hide
        Chris Nauroth added a comment -

        Thanks, Haohui. The patch looks good. Would it be possible to add a test that runs the XML processor on an fsimage containing snapshots, and then asserts that the output is well-formed XML?

        Show
        Chris Nauroth added a comment - Thanks, Haohui. The patch looks good. Would it be possible to add a test that runs the XML processor on an fsimage containing snapshots, and then asserts that the output is well-formed XML?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12630112/HDFS-5981.001.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6193//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6193//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630112/HDFS-5981.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6193//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6193//console This message is automatically generated.
        Hide
        Haohui Mai added a comment -

        The v2 patch addresses the comments from Chris.

        Show
        Haohui Mai added a comment - The v2 patch addresses the comments from Chris.
        Haohui Mai made changes -
        Attachment HDFS-5981.002.patch [ 12630155 ]
        Hide
        Haohui Mai added a comment -

        The v3 patch creates an snapshot in the unit tests. It also removes a redundant println statement.

        Show
        Haohui Mai added a comment - The v3 patch creates an snapshot in the unit tests. It also removes a redundant println statement.
        Haohui Mai made changes -
        Attachment HDFS-5981.003.patch [ 12630158 ]
        Hide
        Chris Nauroth added a comment -

        +1 for the patch. Thank you for adding the test. I'll commit this later today.

        Show
        Chris Nauroth added a comment - +1 for the patch. Thank you for adding the test. I'll commit this later today.
        Chris Nauroth made changes -
        Affects Version/s 3.0.0 [ 12320356 ]
        Affects Version/s 2.4.0 [ 12326143 ]
        Target Version/s 3.0.0, 2.4.0 [ 12320356, 12326143 ]
        Component/s tools [ 12312944 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12630155/HDFS-5981.002.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.fs.loadGenerator.TestLoadGenerator

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6195//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6195//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630155/HDFS-5981.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.loadGenerator.TestLoadGenerator +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6195//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6195//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12630158/HDFS-5981.003.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
        org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6196//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6196//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630158/HDFS-5981.003.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.loadGenerator.TestLoadGenerator org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6196//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6196//console This message is automatically generated.
        Hide
        Haohui Mai added a comment -

        HDFS-5991 is tracking the failure of TestLoadGenerator. The failure of TestCacheDirectives is unrelated.

        Show
        Haohui Mai added a comment - HDFS-5991 is tracking the failure of TestLoadGenerator . The failure of TestCacheDirectives is unrelated.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5202 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5202/)
        HDFS-5981. loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5202 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5202/ ) HDFS-5981 . loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Chris Nauroth added a comment -

        I committed this patch to trunk, branch-2 and branch-2.4. Thank you for the patch, Haohui Mai,.

        Show
        Chris Nauroth added a comment - I committed this patch to trunk, branch-2 and branch-2.4. Thank you for the patch, Haohui Mai ,.
        Chris Nauroth made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 3.0.0 [ 12320356 ]
        Fix Version/s 2.4.0 [ 12326143 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #488 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/488/)
        HDFS-5981. loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #488 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/488/ ) HDFS-5981 . loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1680 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1680/)
        HDFS-5981. loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1680 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1680/ ) HDFS-5981 . loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1705 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1705/)
        HDFS-5981. loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1705 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1705/ ) HDFS-5981 . loadGenerator exit code is not reliable. Contributed by Haohui Mai. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1570468 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Allen Wittenauer made changes -
        Fix Version/s 3.0.0 [ 12320356 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        1m 35s 1 Haohui Mai 20/Feb/14 02:40
        Patch Available Patch Available Resolved Resolved
        1d 4h 13m 1 Chris Nauroth 21/Feb/14 06:54
        Resolved Resolved Closed Closed
        48d 6h 16m 1 Arun C Murthy 10/Apr/14 14:11

          People

          • Assignee:
            Haohui Mai
            Reporter:
            Haohui Mai
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development