[HIVE-18440] CombineHiveInputFormat fails with IllegalArgumentException when the path is empty

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.2.1
• Component/s: File Formats
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 11/Jan/18 06:56
• Updated: 11/Jan/18 11:02
• Resolved: 11/Jan/18 11:02

Description

Hive query failed with IllegalArgumentException when the number of paths is zero. We create the fixedThreadPool with same value as number of paths and it should be greater than zero.

2018-01-05 00:32:55,724 [INFO] [InputInitializer {Map 14} #0] |io.CombineHiveInputFormat|: Total number of paths: 0, launching 0 threads to check non-combinable ones.

2018-01-05 00:32:55,745 [ERROR] [Dispatcher thread {Central}] |impl.VertexImpl|: Vertex Input: share_of_shelf_kpi_values_int_01_001 initializer failed, vertex=vertex_1512053476937_11160_1_01 [Map 14]
org.apache.tez.dag.app.dag.impl.AMUserCodeException: java.lang.IllegalArgumentException
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallback.onFailure(RootInputInitializerManager.java:319)
        at com.google.common.util.concurrent.Futures$4.run(Futures.java:1140)
        at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
        at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:150)
        at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:135)
        at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91)
        at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
        at java.util.concurrent.FutureTask.setException(FutureTask.java:251)
        at java.util.concurrent.FutureTask.run(FutureTask.java:271)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1307)
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1195)
        at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89)
        at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:519)
        at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:447)
        at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:299)
        at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:273)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:266)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:266)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)