[HDFS-14312] KMS-o-meter: Scale test KMS using kms audit log

• Type: New Feature
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: kms
• Target Version/s: 3.5.0
• Assignee: Wei-Chiu Chuang
• Reporter: Wei-Chiu Chuang
• Created: 22/Feb/19 17:32
• Updated: 04/Jan/24 08:17

Description

It appears to me that Dynamometer's architecture allows KMS scale tests too.

I imagine there are two ways to scale test a KMS.

1. Take KMS audit logs, and replay the logs against a KMS.
2. Configure Dynamometer to start KMS in addition to NameNode. Assuming the fsimage comes from an encrypted cluster, replaying HDFS audit log also tests KMS.

It would be even more interesting to have a tool that converts uncrypted cluster fsimage to an encrypted one.

Issue Links

• is related to: HADOOP-16284 KMS Cache Miss Storm
• is related to: HADOOP-16119 KMS on Hadoop RPC Engine