[ACCUMULO-578] use hdfs for the walog

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: logger, tserver
• Assignee: Eric C. Newton
• Reporter: Eric C. Newton
• Created: 07/May/12 14:47
• Updated: 23/Apr/13 01:21
• Resolved: 05/Jun/12 13:21

Description

Using HDFS for walogs would fix:

• ACCUMULO-84: any node can read the replicated files
• ACCUMULO-558: wouldn't need to monitor loggers
• ACCUMULO-544: log references wouldn't include hostnames
• ACCUMULO-423: wouldn't need to monitor loggers
• ACCUMULO-258: hdfs has load balancing already

To implement it, we would need the ability to distribute log sorts.

Continuing to use loggers helps us avoid:

• hdfs pipeline strategy
• we don't have fine-grained insight when a single node makes dfs slow
• additional namenode pressure
• flexibility: for example, we can add fadvise() calls to the logger before HDFS supports it