[HBASE-26763] Setback occurs to AsyncFSWAL#highestProcessedAppendTxid and then IllegalArgumentException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0-alpha-2, 2.4.9
• Component/s: wal
• Assignee: Unassigned
• Reporter: Xiaolin Ha
• Created: 21/Feb/22 03:54
• Updated: 22/Feb/22 09:56
• Resolved: 22/Feb/22 09:56

Description

The consumer thread in AsyncFSWAL crashed by IllegalArgumentException, stack trace is as follows,

Exception in thread "XXXX" java.lang.IllegalArgumentException: fromKey > toKey
        at java.util.TreeMap$NavigableSubMap.<init>(TreeMap.java:1368)
        at java.util.TreeMap$AscendingSubMap.<init>(TreeMap.java:1855)
        at java.util.TreeMap.subMap(TreeMap.java:913)
        at java.util.TreeSet.subSet(TreeSet.java:325)
        at java.util.TreeSet.subSet(TreeSet.java:361)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.isHsync(AsyncFSWAL.java:375)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:392)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.appendAndSync(AsyncFSWAL.java:504)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:565)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Exception in thread "XXXX" java.lang.IllegalArgumentException: fromKey > toKey
        at java.util.TreeMap$NavigableSubMap.<init>(TreeMap.java:1368)
        at java.util.TreeMap$AscendingSubMap.<init>(TreeMap.java:1855)
        at java.util.TreeMap.subMap(TreeMap.java:913)
        at java.util.TreeSet.subSet(TreeSet.java:325)
        at java.util.TreeSet.subSet(TreeSet.java:361)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.isHsync(AsyncFSWAL.java:375)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:392)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.appendAndSync(AsyncFSWAL.java:504)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:565)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745) 

Issue Links

• is duplicated by: HBASE-25905 Shutdown of WAL stuck at waitForSafePoint