[ACCUMULO-2110] [RW] Error in Bulk.Verify

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: test
• Labels: 16_qa_bug
• Assignee: Eric C. Newton
• Reporter: Josh Elser
• Created: 29/Dec/13 04:12
• Updated: 13/Mar/14 16:07
• Resolved: 10/Jan/14 20:11

Description

Saw this across three different RW clients. Obviously the key differs each time.

java.lang.Exception: Error running node Bulk.xml
        at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:285)
        at org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:65)
        at org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:125)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.accumulo.start.Main$1.run(Main.java:137)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.Exception: Error running node bulk.Verify
        at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:285)
        at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:254)
        ... 8 more
Caused by: java.lang.Exception: Bad key at r08930 cf:000 [] 1388209514232 false 1
        at org.apache.accumulo.test.randomwalk.bulk.Verify.visit(Verify.java:65)
        at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:254)
        ... 9 more

Some relevant logs from the RW client, but there's nothing in the server log that I've seen.

27 21:18:34,660 [bulk.BulkPlusOne] DEBUG: preparing bulk files with start rows [r00000, r00483, r08930, r0a413, r0a603, r0b20e, r10a31, r1481e, r15e3d, r1853e] last row r1869f marker 0000379
...
27 21:21:03,710 [bulk.BulkPlusOne] DEBUG: Finished bulk import, start rows [r00000, r00483, r08930, r0a413, r0a603, r0b20e, r10a31, r1481e, r15e3d, r1853e] last row r1869f marker 0000379
27 21:21:03,710 [bulk.BulkPlusOne] INFO : Incrementing
...
27 21:48:36,260 [impl.ThriftTransportPool] INFO : Thread "bulkImportPool 3" no longer stuck on IO  to master:9999 (0) sawError = false
27 21:48:36,347 [bulk.Compact] INFO : Compaction (r0349f -> r08a2e] finished
27 21:48:44,229 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r0077a;r004a1,tserver1:9997,1432840570f049e)
27 21:48:46,145 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r007d5;r0077a,tserver1:9997,1432840570f049e) 
27 21:48:48,738 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r00f3b;r00e54,tserver3:9997,34328404fc00428)
27 21:48:50,331 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r014f1;r00f3b,tserver3:9997,34328404fc00428)
27 21:49:00,030 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r017a3;r014f1,tserver1:9997,1432840570f049e)
27 21:49:07,122 [impl.ThriftScanner] DEBUG: Scan failed, not serving tablet (lw;r02f24;r02824,tserver4:9997,1432840570f045b)

Issue Links

• is related to: ACCUMULO-2466 Bulk randomwalk fails with bad key