[ACCUMULO-3303] funky performance with large WAL

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.6.1
• Component/s: logger, tserver
• Assignee: Unassigned
• Reporter: Adam Fuchs
• Created: 05/Nov/14 23:20
• Updated: 08/Dec/14 15:53
• Resolved: 07/Nov/14 22:43

Description

The tserver seems to get into a funky state when writing to a large write-ahead log. I ran some continuous ingest tests varying tserver.walog.max.size in

{512M, 1G, 2G, 4G, 8G}

and got some results that I have yet to understand. I was expecting to see the effects of walog metadata management as described in ACCUMULO-2889, but I also found an additional behavior of ingest slowing down for long periods when using a large walog size.

The cluster configuration was as follows:

Accumulo version: 1.6.2-SNAPSHOT (current head of origin/1.6)
Nodes: 4
Masters: 1
Slaves: 3
Cores per node: 24
Drives per node: 8x1TB data + 2 raided system
Memory per node: 64GB
tserver.memory.maps.max=2G
table.file.compress.type=snappy (for ci table only)
tserver.mutation.queue.max=16M
tserver.wal.sync.method=hflush
Native maps enabled

Issue Links

• depends upon: HDFS-7380 unsteady and slow performance when writing to file with block size >2GB