Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5016

Local mode FileNotFoundException: File does not exist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Not A Problem
    • Affects Version/s: 0.10.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:
    • Release Note:
      This change will allow hive to operate in local mode.

      Description

      Hive jobs in local mode fail with the error posted below. The jar file that's not being found exists and has the following access:
      > ls -l hive-0.10.0/lib/hive-builtins-0.10.0.jar
      rw-rw-r-- 1 ashahab ashahab 3914 Dec 18 2012 hive-0.10.0/lib/hive-builtins-0.10.0.jar

      Steps to reproduce:

      hive> set hive.exec.mode.local.auto=true;
      hive> set hive.exec.mode.local.auto;
      hive.exec.mode.local.auto=true
      hive> select count from abin_test_table;
      Automatically selecting local only mode for query
      Total MapReduce jobs = 1
      Launching Job 1 out of 1
      Number of reduce tasks determined at compile time: 1
      In order to change the average load for a reducer (in bytes):
      set hive.exec.reducers.bytes.per.reducer=<number>
      In order to limit the maximum number of reducers:
      set hive.exec.reducers.max=<number>
      In order to set a constant number of reducers:
      set mapred.reduce.tasks=<number>
      13/08/06 21:37:11 WARN conf.Configuration: file:/tmp/ashahab/hive_2013-08-06_21-37-09_046_3263640403676309186/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
      13/08/06 21:37:11 WARN conf.Configuration: file:/tmp/ashahab/hive_2013-08-06_21-37-09_046_3263640403676309186/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
      WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
      Execution log at: /tmp/ashahab/ashahab_20130806213737_7d26b796-5f55-44ca-a755-8898153d963b.log
      java.io.FileNotFoundException: File does not exist: /home/ashahab/dev/hive-0.10.0/lib/hive-builtins-0.10.0.jar
      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:782)
      at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:208)
      at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:71)
      at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:252)
      at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:290)
      at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:361)
      at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
      at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
      at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:617)
      at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:612)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:612)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:689)

        Attachments

        1. HIVE-5016.patch
          0.9 kB
          Abin Shahab
        2. HIVE-5016.patch
          2 kB
          Abin Shahab

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ashahab Abin Shahab
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: