Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24245

Failed to write replication wal position

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Information Provided
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.4.10
    • Component/s: Replication
    • Labels:
      None
    • Environment:

      apache-phoenix-4.15.0-HBase-1.5

      hadoop-2.10.0

      hbase-1.5.0-bin

      zookeeper-3.4.14

      Description

      I turned on replication for tables, index tables did not, wrote data using phoneix, and hbase regionserver went down after running for a while.

      regionserver log:

      2020-04-20 19:22:49,531 FATAL [main-EventThread.replicationSource,1.replicationSource.a2-test-regionserver-52-23.sh%2C16020%2C1587378758523,1] regionserver.HRegionServe
      r: ABORTING region server a2- test-regionserver-52-23.sh,16020,1587378758523: Failed to write replication wal position (filename=a2-test-regionserver-52-23.sh%2C16020%2C
      1587378758523.1587381739689, position=3388929)
      org.apache.zookeeper. KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase/replication/rs/a2-test-regionserver-52-23.sh,16020,1587378758523/1/a2-test-reg
      ionserver-52-23.sh%2C16020%2C1587378758523. 1587381739689
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
      at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1327)
      at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.setData(RecoverableZooKeeper.java:510)
      at org.apache.hadoop.hbase.zookeeper.ZKUtil .setData(ZKUtil.java:867)
      at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:917)
      at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:911 )
      at org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl.setLogPosition(ReplicationQueuesZKImpl.java:155)
      at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:197)
      at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.updateLogPosition(ReplicationSource.java:791)
      at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.shipEdits(ReplicationSource.java:762)
      at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.run(ReplicationSource.java:601)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Jread L luodx
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: