[HDFS-3025] Automatic log sync shouldn't happen inside logEdit path

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.23.3
• Component/s: namenode, performance
• Target Version/s: 2.0.0-alpha
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 28/Feb/12 19:37
• Updated: 18/Apr/12 20:01

Description

HDFS-3020 fixes the "automatic log sync" functionality so that, when logEdits is called without log sync, it eventually triggers a sync. That sync ends up being inline, though, which means the FSN lock is usually held during it. This causes a bunch of threads to pile up.

Instead, we should have it just set a "syncNeeded" flag and trigger a sync from another thread which isn't holding the lock (or from the same thread using a "logSyncIfNeeded" call).

(credit to the FB branch for this idea)