[HIVE-17462] hive_1.2.1 hiveserver2 memory leak

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.2.1
• Fix Version/s: 1.2.1

Environment

hive version 1.2.1

• Assignee: kongxianghe
• Reporter: gehaijiang
• Created: 06/Sep/17 12:14
• Updated: 21/Sep/20 07:15
• Resolved: 21/Sep/20 07:15

Description

hiveserver2 memory leak

hive use third UDF （vs-1.0.2-SNAPSHOT.jar ， alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar ..... and so on )

lr-x------ 1 data data 64 Sep 5 18:37 964 -> /tmp/9e38cc04-5693-474b-9c7d-bfdd978bcbb4_resources/vs-1.0.2-SNAPSHOT.jar (deleted)
lr-x------ 1 data data 64 Sep 6 10:41 965 -> /tmp/188bbf2a-d8a5-48a7-81fc-b807f9ff201d_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)
lr-x------ 1 data data 64 Sep 6 17:41 97 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/jsr305-3.0.0.jar
lrwx------ 1 data data 64 Sep 5 18:37 975 -> socket:[1318353317]
lr-x------ 1 data data 64 Sep 6 02:38 977 -> /tmp/64e309dc-352f-4ba4-b871-1aa78fe05945_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)
lr-x------ 1 data data 64 Sep 6 17:41 98 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar
lrwx------ 1 data data 64 Sep 6 08:40 983 -> socket:[1299459344]
lr-x------ 1 data data 64 Sep 5 19:37 987 -> /tmp/c3054987-c9c6-468a-8b5c-6e20b1972e0b_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)
lr-x------ 1 data data 64 Sep 6 17:41 99 -> /home/data/programs/hadoop-2.7.1/share/hadoop/hdfs/lib/guava-11.0.2.jar
lr-x------ 1 data data 64 Sep 6 08:40 994 -> /tmp/fc5c44b3-9bd8-4a32-a39a-66cd44032fee_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)
lr-x------ 1 data data 64 Sep 6 06:39 996 -> /tmp/3b3c2bd6-0a0e-4599-b757-4a048a968457_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)
lr-x------ 1 data data 64 Sep 5 17:36 999 -> /tmp/6ad76494-cdda-430b-b7d0-2213731655a8_resources/alogdata-1.0.3-SNAPSHOT-jar-with-dependencies.jar (deleted)

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20084 data 20 0 13.6g 11g 533m S 62.3 9.2 6619:16 java

/home/data/programs/jdk/jdk-current/bin/java-Djava.net.preferIPv4Stack=true-Dhadoop.log.dir=/home/data/hadoop/logs-Dhadoop.log.file=hadoop.log-Dhadoop.home.dir=/home/data/programs/hadoop-2.7.1-Dhadoop.id.str=data-Dhadoop.root.logger=INFO,DRFA-Djava.library.path=/home/data/programs/hadoop-2.7.1/lib/native-Dhadoop.policy.file=hadoop-policy.xml-Djava.net.preferIPv4Stack=true-XX:+UseConcMarkSweepGC-Xms8g-Xmx8g-Dhadoop.security.logger=INFO,NullAppenderorg.apache.hadoop.util.RunJar/home/data/programs/hive-current/lib/hive-service-1.2.1.jarorg.apache.hive.service.server.HiveServer2--hiveconfhive.log.file=hiveserver2.log

Issue Links

• is fixed by: HIVE-10453 HS2 leaking open file descriptors when using UDFs