[HADOOP-17003] No Log compression and retention in Hadoop

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: kms
• Assignee: Ahmed Hussein
• Reporter: Ahmed Hussein
• Created: 20/Apr/20 16:15
• Updated: 16/Jul/20 15:25

Description

Hadoop logging lacks several important features. Logs generated end up eating disk space
We need an implementation that satisfies the following three features: 1) time-based rolling, 2) retention and 3) compression.

For example KMS logs have no retention or compression.

-rw-r--r-- 1 hkms users 704M Mar 20 23:59 kms.log.2020-03-20
-rw-r--r-- 1 hkms users 731M Mar 21 23:59 kms.log.2020-03-21
-rw-r--r-- 1 hkms users 750M Mar 22 23:59 kms.log.2020-03-22
-rw-r--r-- 1 hkms users 757M Mar 23 23:59 kms.log.2020-03-23
-rw-r--r-- 1 hkms users 805M Mar 24 23:59 kms.log.2020-03-24
-rw-r--r-- 1 hkms users 858M Mar 25 23:59 kms.log.2020-03-25
-rw-r--r-- 1 hkms users 875M Mar 26 23:59 kms.log.2020-03-26
-rw-r--r-- 1 hkms users 754M Mar 27 23:59 kms.log.2020-03-27

Issue Links

• is blocked by: HADOOP-16206 Migrate from Log4j1 to Log4j2