[BOOKKEEPER-916] Placement policy to accomodate different types of ledger storage

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.3.2
• Component/s: bookkeeper-client, bookkeeper-server
• Labels: ghmigrated iktriage0
• Assignee: Rithin Shetty
• Reporter: JV Jujjuri
• Created: 07/Apr/16 06:10
• Updated: 17/Oct/17 21:33

Description

As we start to use bookkeeper as long term storage, it may not be right use of resources to keep all copies of entry (write ensemble) on efficient storage. This feature is to come up with an intelligent placement that distributes entry copies across different classes of storage.

Simply put, say we have SSD based ledger storage and HDD based ledger storage on each system. Instead of putting all copies of entries either on SSD or on HDD, this placement policy maintains one copy on SSD and others on HDD.

• Have at least one copy on SSD and others on HDD.
• Writer need to be aware of this classification
• Replication logic need to be aware of this logic.

• While reading attempt to read from SSD first.
• Reader also need to be aware of this logic.

This will push bookkeeper towards the long term storage, also can be a stepping store towards introducing storage tiers in the future.

This has dependency/relation to

https://issues.apache.org/jira/browse/BOOKKEEPER-912.
https://issues.apache.org/jira/browse/BOOKKEEPER-915

Issue Links

• is related to: BOOKKEEPER-915 Auto replication should honor ensemble placement policy
• is related to: BOOKKEEPER-912 Allow EnsemblePlacementPolicy to choose bookies using ledger custom data (multitenancy support)