-
Type:
Sub-task
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 3.1.0
-
Component/s: None
-
Labels:None
-
Hadoop Flags:Reviewed
The design of Provided Storage requires the use of an AliasMap to manage the mapping between blocks of files on the local HDFS and ranges of files on a remote storage system. To reduce load from the Namenode, this can be done using a pluggable external service (e.g. AzureTable, Cassandra, Ratis). However, to aide adoption and ease of deployment, we propose an in memory version.
This AliasMap will be a wrapper around LevelDB (already a dependency from the Timeline Service) and use protobuf for the key (blockpool, blockid, and genstamp) and the value (url, offset, length, nonce). The in memory service will also have a configurable port on which it will listen for updates from Storage Policy Satisfier (SPS) Coordinating Datanodes (C-DN).
- depends upon
-
HDFS-11902 [READ] Merge BlockFormatProvider and FileRegionProvider.
-
- Resolved
-
- is depended upon by
-
HDFS-12591 [READ] Implement LevelDBFileRegionFormat
-
- Resolved
-
- is related to
-
HDFS-13795 Fix potential NPE in InMemoryLevelDBAliasMapServer
-
- Resolved
-
-
HDFS-13536 [PROVIDED Storage] HA for InMemoryAliasMap
-
- Resolved
-
-
HDFS-13796 Allow verbosity of InMemoryLevelDBAliasMapServer to be configurable
-
- Resolved
-
-
HDFS-13798 Create InMemoryAliasMap location if missing.
-
- Patch Available
-
-
HDFS-12944 Update NOTICE for AssertJ dependency
-
- Resolved
-
- requires
-
HDFS-12713 [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata
-
- Resolved
-