[OOZIE-3396] AuthOozieClient leaks memory

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.0.0
• Assignee: Junfan Zhang
• Reporter: Junfan Zhang
• Created: 03/Dec/18 08:06
• Updated: 05/Dec/18 10:51

Description

Phenomenon:
Recently we upgraded the AuthOozieClient and added the OOZIE-2447. When we were running online, the server memory usage continued to increase. Through the jmap and jhat command analysis, it was found that the deleteOnExist method in the writeAuthToken method under the AuthOozieClient class occupies a large amount of memory, which is caused by OOZIE-2447. Also, because I have applied the OOZIE-3379 in the production environment. I still find that the .oozie-auth-token-xxxx file update time has changed, but the file content has not changed.

After reviewing the code, I found that currentToken.equals(readToken) in the AuthOozieClient class is always false because the AuthenticatedURL.token does not have an equals method. The reason is because currentToken.equals(readToken) is always false, causing the tmp file to be created continuously, but there is a memory leak due to the deleteOnExist method (https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook)

Solution:
1. In AuthOozieClient, change to currentToken.toString().equals(readToken.toString())
2. In addition, the deleteOnExist method should be changed to the delete() in AuthOozieClient class.

Doubt:
Why delete the equals method in the AuthenticatedURL.Token class in Hadoop? See: https://issues.apache.org/jira/browse/HADOOP-10771

Issue Links

• relates to: OOZIE-3379 [client] Auth token cache file name should include OOZIE_URL