Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12828

OIV ReverseXML Processor fails with escaped characters

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.8.5
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The HDFS OIV ReverseXML processor fails if the XML file contains escaped characters:

      ekrogen at ekrogen-ld1 in ~/dev/hadoop/trunk/hadoop-dist/target/hadoop-3.0.0-beta1-SNAPSHOT on trunk!
      ± $HADOOP_HOME/bin/hdfs dfs -fs hdfs://localhost:9000/ -ls /
      Found 4 items
      drwxr-xr-x   - ekrogen supergroup          0 2017-11-16 14:48 /foo
      drwxr-xr-x   - ekrogen supergroup          0 2017-11-16 14:49 /foo"
      drwxr-xr-x   - ekrogen supergroup          0 2017-11-16 14:50 /foo`
      drwxr-xr-x   - ekrogen supergroup          0 2017-11-16 14:49 /foo&
      

      Then after doing saveNamespace on that NameNode...

      ekrogen at ekrogen-ld1 in ~/dev/hadoop/trunk/hadoop-dist/target/hadoop-3.0.0-beta1-SNAPSHOT on trunk!
      ± $HADOOP_HOME/bin/hdfs oiv -i /tmp/hadoop-ekrogen/dfs/name/current/fsimage_0000000000000000008 -o /tmp/hadoop-ekrogen/dfs/name/current/fsimage_0000000000000000008.xml -p XML
      ekrogen at ekrogen-ld1 in ~/dev/hadoop/trunk/hadoop-dist/target/hadoop-3.0.0-beta1-SNAPSHOT on trunk!
      ± $HADOOP_HOME/bin/hdfs oiv -i /tmp/hadoop-ekrogen/dfs/name/current/fsimage_0000000000000000008.xml -o /tmp/hadoop-ekrogen/dfs/name/current/fsimage_0000000000000000008.xml.rev -p ReverseXML
      OfflineImageReconstructor failed: unterminated entity ref starting with &
      org.apache.hadoop.hdfs.util.XMLUtils$UnmanglingError: unterminated entity ref starting with &
              at org.apache.hadoop.hdfs.util.XMLUtils.unmangleXmlString(XMLUtils.java:232)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.loadNodeChildrenHelper(OfflineImageReconstructor.java:383)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.loadNodeChildrenHelper(OfflineImageReconstructor.java:379)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.loadNodeChildren(OfflineImageReconstructor.java:418)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.access$1000(OfflineImageReconstructor.java:95)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeSectionProcessor.process(OfflineImageReconstructor.java:524)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processXml(OfflineImageReconstructor.java:1710)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(OfflineImageReconstructor.java:1765)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.run(OfflineImageViewerPB.java:191)
              at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.main(OfflineImageViewerPB.java:134)
      

      See attachments for relevant fsimage XML file.

        Attachments

        1. HDFS-12828.000.patch
          2 kB
          Erik Krogen
        2. fsimage_0000000000000000008.xml
          2 kB
          Erik Krogen

          Activity

            People

            • Assignee:
              xkrogen Erik Krogen
              Reporter:
              xkrogen Erik Krogen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: