[HIVE-10453] HS2 leaking open file descriptors when using UDFs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.1, 1.3.0, 2.0.0
• Component/s: UDF
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 23/Apr/15 00:53
• Updated: 21/Sep/20 07:06
• Resolved: 22/Jun/15 08:47

Description

1. create a custom function by
CREATE FUNCTION myfunc AS 'someudfclass' using jar 'hdfs:///tmp/myudf.jar';
2. Create a simple jdbc client, just do
connect,
run simple query which using the function such as:
select myfunc(col1) from sometable
3. Disconnect.
Check open file for HiveServer2 by:
lsof -p HSProcID | grep myudf.jar
You will see the leak as:

java      28718 ychen  txt      REG                1,4        741 212977666 /private/var/folders/6p/7_njf13d6h144wldzbbsfpz80000gp/T/1bfe3de0-ac63-4eba-a725-6a9840f1f8d5_resources/myudf.jar
java      28718 ychen  330r     REG                1,4        741 212977666 /private/var/folders/6p/7_njf13d6h144wldzbbsfpz80000gp/T/1bfe3de0-ac63-4eba-a725-6a9840f1f8d5_resources/myudf.jar

Issue Links

• fixes: HIVE-17462 hive_1.2.1 hiveserver2 memory leak
• relates to: HIVE-10329 Hadoop reflectionutils has issues
• relates to: HIVE-6672 JARs loaded by permanent functions don't work properly with HiveServer2
• relates to: HIVE-3969 Session state for hive server should be cleaned-up
• relates to: HIVE-10970 Investigate HIVE-10453: HS2 leaking open file descriptors when using UDFs
• mentioned in: Wiki Page Loading...