Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18440

CombineHiveInputFormat fails with IllegalArgumentException when the path is empty

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.2.1
    • Fix Version/s: None
    • Component/s: File Formats
    • Labels:
      None

      Description

      Hive query failed with IllegalArgumentException when the number of paths is zero. We create the fixedThreadPool with same value as number of paths and it should be greater than zero.

      
      2018-01-05 00:32:55,724 [INFO] [InputInitializer {Map 14} #0] |io.CombineHiveInputFormat|: Total number of paths: 0, launching 0 threads to check non-combinable ones.
      
      2018-01-05 00:32:55,745 [ERROR] [Dispatcher thread {Central}] |impl.VertexImpl|: Vertex Input: share_of_shelf_kpi_values_int_01_001 initializer failed, vertex=vertex_1512053476937_11160_1_01 [Map 14]
      org.apache.tez.dag.app.dag.impl.AMUserCodeException: java.lang.IllegalArgumentException
              at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallback.onFailure(RootInputInitializerManager.java:319)
              at com.google.common.util.concurrent.Futures$4.run(Futures.java:1140)
              at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
              at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:150)
              at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:135)
              at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91)
              at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
              at java.util.concurrent.FutureTask.setException(FutureTask.java:251)
              at java.util.concurrent.FutureTask.run(FutureTask.java:271)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.IllegalArgumentException
              at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1307)
              at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1195)
              at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89)
              at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:519)
              at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:447)
              at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:299)
              at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
              at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:273)
              at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:266)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
              at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:266)
              at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              prabhujoseph Prabhu Joseph Assign to me
              Reporter:
              prabhujoseph Prabhu Joseph

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment