Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3137

Spark cubing without hive-site.xml

    XMLWordPrintableJSON

    Details

      Description

      Getting following exception while trying to build a cube

      java.lang.RuntimeException: Cannot find hive-site.xml in kylin_hadoop_conf_dir: /etc/hadoop/conf. In order to enable spark cubing, you must set kylin.env.hadoop-conf-dir to a dir which contains at least core-site.xml, hdfs-site.xml, hive-site.xml, mapred-site.xml, yarn-site.xml
      	at org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:117)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
      	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
      	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:144)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

      I am using Kylin binaries for CDH downloaded from kylin.apache.org.
      Yes, indeed hive-site.xml is not in /etc/hadoop/conf in Cloudera's distribution for Hadoop.

      hive-site.xml is in /etc/hive/conf, not in /etc/hadoop/conf

      The other four files:
      core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml
      can be found in /etc/hadoop/conf but, again, not hive-site.xml which is in /etc/hive/conf .

      Would be great to have this adjusted for CDH.

        Attachments

          Activity

            People

            • Assignee:
              shaofengshi Shao Feng Shi
              Reporter:
              Tagar Ruslan Dautkhanov
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: