Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In doing testing and benchmarking of the NameNode and dependent systems, it's often useful to be able to use an fsimage provided by some production system in a controlled environment without actually having access to any of the data. For example, while doing some recent work on Apache Impala I was trying to optimize the transmission and storage of block locations and tokens and measure the results based on metadata from a production user. In order to achieve this, it would be useful for the NN to expose a developer-only (undocumented) configuration to generate fake block locations and return them to callers. The "fake" locations should be randomly distributed across a fixed set of fake datanodes.