Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18379

SnapshotManager#checkSnapshotSupport() should better handle malfunctioning hdfs snapshot

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      The following was observed by a customer which prevented master from coming up:

      2017-07-13 13:25:07,898 FATAL [xyz:16000.activeMasterManager] master.HMaster: Failed to become active master
      java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: Daily_Snapshot_Apps_2017-xx
      at org.apache.hadoop.fs.Path.initialize(Path.java:205)
      at org.apache.hadoop.fs.Path.<init>(Path.java:171)
      at org.apache.hadoop.fs.Path.<init>(Path.java:93)
      at org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:230)
      at org.apache.hadoop.hdfs.protocol.HdfsFileStatus.makeQualified(HdfsFileStatus.java:263)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:911)
      at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
      at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
      at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
      at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1534)
      at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1574)
      at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.getCompletedSnapshots(SnapshotManager.java:206)
      at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.checkSnapshotSupport(SnapshotManager.java:1011)
      at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.initialize(SnapshotManager.java:1070)
      at org.apache.hadoop.hbase.procedure.MasterProcedureManagerHost.initialize(MasterProcedureManagerHost.java:50)
      at org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:667)
      at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:732)
      at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:213)
      at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1863)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.net.URISyntaxException: Relative path in absolute URI: Daily_Snapshot_Apps_2017-xx
      at java.net.URI.checkPath(URI.java:1823)
      at java.net.URI.<init>(URI.java:745)
      at org.apache.hadoop.fs.Path.initialize(Path.java:202)
      

      Turns out the exception can be reproduced using hdfs command line accessing .snapshot directory.

      SnapshotManager#checkSnapshotSupport() should better handle malfunctioning hdfs snapshot so that master starts up.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yuzhihong@gmail.com Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: