[HIVE-20824] HiveServer2 is still leaking file handles when using ADD JAR

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.1.1, 2.3.2
• Component/s: HiveServer2
• Assignee: Unassigned
• Reporter: hezhang
• Created: 27/Oct/18 04:38
• Updated: 28/Oct/18 14:44

Description

In Hive version 2.1.1, hiveserver2 is stll leaking jar file handle when using add jar . When using UDF jar and  creating function with this jar,  the jar file will not delete.

Beeline version 2.1.1 by Apache Hive
0: jdbc:hive2://localhost:7001/> add jar hdfs:///udf/UDF-1.0-SNAPSHOT.jar;No rows affected (0.45 seconds)
0: jdbc:hive2://localhost:7001/> create temporary function my_md5 as 'com.neu.hive.UDF.Md5';No rows affected (0.082 seconds)
0: jdbc:hive2://localhost:7001/> select my_md5(name) from test;+-----------------------------------+--+
| c0 |
+-----------------------------------+--+
| ad6f6c09bd4accc056568714692ff4cd |
| 836db9ecc83d8397a5d0205eb8344d4c |
| 5a7fcd4f1c785c8ef4931a5a9c698ac0 |
| 0e2bb4d743f2a009d4b84a9338c98f7c |
| 3f6697692f4506cf311c95848f3536d3 |
| 0de5fc94d0ba53fc7a44f0f136e82fbb |
| 04ef3e58a6b569c1a404efb90d50f906 |
| f505e0a965c8e721eef7e111c66e5c29 |
| c053092216d6f623615d5be978b98e67 |
| 8c2c58079bc3d10be0da79c5d9940f31 |
+-----------------------------------+--+
10 rows selected (0.12 seconds)

the hiveserver2 PID is 3856，then list the file handle.

[hadoop@10 ~]$ lsof -p 3856 | grep SNAPSHOT.jar
java 3856 hadoop DEL REG 252,16 263184 /data/emr/hive/tmp/2f3bc7b2-c385-4552-a54e-b575b4338d63_resources/UDF-1.0-SNAPSHOT.jar
java 3856 hadoop DEL REG 252,16 263178 /data/emr/hive/tmp/986015db-52bf-4f6a-99f8-977de471d528_resources/UDF-4.0-SNAPSHOT.jar
java 3856 hadoop 774r REG 252,16 90885742 263178 /data/emr/hive/tmp/986015db-52bf-4f6a-99f8-977de471d528_resources/UDF-4.0-SNAPSHOT.jar (deleted)
java 3856 hadoop 775r REG 252,16 104220075 263184 /data/emr/hive/tmp/2f3bc7b2-c385-4552-a54e-b575b4338d63_resources/UDF-1.0-SNAPSHOT.jar (deleted)

 

 

 

 

Issue Links

• Blocked: HIVE-16455 ADD JAR command leaks JAR Files
• Blocked: HIVE-11878 ClassNotFoundException can possibly occur if multiple jars are registered one at a time in Hive