Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7290 Online snapshots
  3. HBASE-7344

subprocedure initialization fails with invalid znode data.

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Sometimes snapshots subprocedures fail to start on RS because data read from ZK is bad.

      2012-12-13 07:22:55,238 ERROR org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs: Illegal argument exception
      java.lang.IllegalArgumentException: Could not read snapshot information from request.
              at org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedureBuilder.buildSubprocedure(RegionServerSnapsh
      otManager.java:284)
              at org.apache.hadoop.hbase.procedure.ProcedureMember.createSubprocedure(ProcedureMember.java:98)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.startNewSubprocedure(ZKProcedureMemberRpcs.java:199)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.waitForNewProcedures(ZKProcedureMemberRpcs.java:167)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.access$1(ZKProcedureMemberRpcs.java:150)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs$1.nodeChildrenChanged(ZKProcedureMemberRpcs.java:106)
              at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:303)
              at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
              at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
      2012-12-13 07:22:55,239 ERROR org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs: Failed due to null subprocedure
      Local ForeignThreadException from null
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.startNewSubprocedure(ZKProcedureMemberRpcs.java:203)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.waitForNewProcedures(ZKProcedureMemberRpcs.java:167)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.access$1(ZKProcedureMemberRpcs.java:150)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs$1.nodeChildrenChanged(ZKProcedureMemberRpcs.java:106)
              at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:303)
              at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
              at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
      Caused by: java.lang.IllegalArgumentException: Could not read snapshot information from request.
              at org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedureBuilder.buildSubprocedure(RegionServerSnapshotManager.java:284)
              at org.apache.hadoop.hbase.procedure.ProcedureMember.createSubprocedure(ProcedureMember.java:98)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.startNewSubprocedure(ZKProcedureMemberRpcs.java:199)
              ... 6 more
      2012-12-13 07:22:55,239 ERROR org.apache.zookeeper.ClientCnxn: Error while calling watcher 
      java.lang.NullPointerException
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.sendMemberAborted(ZKProcedureMemberRpcs.java:266)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.startNewSubprocedure(ZKProcedureMemberRpcs.java:203)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.waitForNewProcedures(ZKProcedureMemberRpcs.java:167)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.access$1(ZKProcedureMemberRpcs.java:150)
              at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs$1.nodeChildrenChanged(ZKProcedureMemberRpcs.java:106)
              at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:303)
              at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
              at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
      

        Attachments

          Activity

            People

            • Assignee:
              jmhsieh Jonathan Hsieh
              Reporter:
              jmhsieh Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: