Currently HDFS does not impose an upper limit on the edit log buffer. In case there are a large number of open operations coming in with access time update on, since open does not call sync automatically, there is a possibility that the buffer grow to a large size, therefore causes memory leak and full GC in extreme cases as described in
The edit log buffer should be automatically flushed when the buffer becomes full.
- relates to
HDFS-3020 Auto-logSync based on edit log buffer size broken