Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4255

Display detailed error message when using livy build error

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Spark Engine
    • Labels:
      None

      Description

      Currently, when using livy build error, the error message does not show the detailed error reason, which is not conducive to troubleshooting. The following is an example of submit build job error:

      Current error message:

      java.lang.RuntimeException: livy execute failed. 
      livy get status failed. state is dead
      	at org.apache.kylin.common.livy.LivyRestExecutor.execute(LivyRestExecutor.java:76)
      	at org.apache.kylin.source.hive.MRHiveDictUtil.runLivySqlJob(MRHiveDictUtil.java:144)
      	at org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.createFlatHiveTable(CreateFlatHiveTableByLivyStep.java:44)
      	at org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.doWork(CreateFlatHiveTableByLivyStep.java:51)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
      	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
      	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

      Actual reason for the error:

      2019-11-13 07:40:02 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      Exception in thread "main" java.io.FileNotFoundException: File hdfs://localhost:9000/kylin/livy/hbase-client-$HBASE_VERSION.jar does not exist.
      	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:697)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751)
      	at org.apache.spark.util.Utils$.fetchHcfsFile(Utils.scala:727)
      	at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:695)
      	at org.apache.spark.deploy.DependencyUtils$.downloadFile(DependencyUtils.scala:135)
      	at org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
      	at org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      

        Attachments

          Activity

            People

            • Assignee:
              codingforfun weibin0516
              Reporter:
              codingforfun weibin0516
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: