[HIVE-25130] alter table concat gives NullPointerException, when data is inserted from Spark

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Kishen Das
• Reporter: Kishen Das
• Created: 18/May/21 00:52
• Updated: 12/Nov/21 21:25
• Resolved: 12/Nov/21 21:25

Description

This is the complete stack trace of the NullPointerException

2021-03-01 14:50:32,201 ERROR org.apache.hadoop.hive.ql.exec.Task: [HiveServer2-Background-Pool: Thread-76760]: Job Commit failed with exception 'java.lang.NullPointerException(null)'

java.lang.NullPointerException

at org.apache.hadoop.hive.ql.exec.Utilities.getAttemptIdFromFilename(Utilities.java:1333)

at org.apache.hadoop.hive.ql.exec.Utilities.compareTempOrDuplicateFiles(Utilities.java:1966)

at org.apache.hadoop.hive.ql.exec.Utilities.ponderRemovingTempOrDuplicateFile(Utilities.java:1907)

at org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFilesNonMm(Utilities.java:1892)

at org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFiles(Utilities.java:1797)

at org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFiles(Utilities.java:1674)

at org.apache.hadoop.hive.ql.exec.Utilities.mvFileToFinalPath(Utilities.java:1544)

at org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.jobCloseOp(AbstractFileMergeOperator.java:304)

at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:798)

at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:637)

at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:335)

at org.apache.hadoop.hive.ql.ddl.table.storage.concatenate.AlterTableConcatenateOperation.executeTask(AlterTableConcatenateOperation.java:129)

at org.apache.hadoop.hive.ql.ddl.table.storage.concatenate.AlterTableConcatenateOperation.execute(AlterTableConcatenateOperation.java:63)

at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80)

at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)

at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)

at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:357)

at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330)

at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)

at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)

at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:740)

at org.apache.hadoop.hive.ql.Driver.run(Driver.java:495)

at org.apache.hadoop.hive.ql.Driver.run(Driver.java:489)

at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)

at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)

at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)

at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)

at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:340)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Issue Links

• is fixed by: HIVE-24936 Fix file name parsing and copy file move.
• links to: GitHub Pull Request #2285