[HBASE-5930] Limits the amount of time an edit can live in the memstore. - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.98.0, 0.94.8, 0.95.1
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed
Release Note:

Hide
This feature limits the time that unflushed data will stay in the memstore.
By default a memstore will flush if data older than 1h (3600000ms) is present.

This can be configured via hbase.regionserver.optionalcacheflushinterval (default value is 3600000).

Show
This feature limits the time that unflushed data will stay in the memstore. By default a memstore will flush if data older than 1h (3600000ms) is present. This can be configured via hbase.regionserver.optionalcacheflushinterval (default value is 3600000).

Description

A colleague of mine ran into an interesting issue.
He inserted some data with the WAL disabled, which happened to fit in the aggregate Memstores memory.

Two weeks later he a had problem with the HDFS cluster, which caused the region servers to abort. He found that his data was lost. Looking at the log we found that the Memstores were not flushed at all during these two weeks.

Should we have an option to flush memstores periodically. There are obvious downsides to this, like many small storefiles, etc.