Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19312

Spark gives wrong error message when failes to create file due to hdfs quota limit.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Not A Problem
    • Affects Version/s: 1.6.0
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:
    • Environment:

      CDH 5.8

      Description

      If we set quota on user space and then will try to create table through hive on spark, which will need more space then avaliable, spark will fail with:

      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException): failed to create file /user/xxxx/hive_db/.hive-staging_hive_..../task_tmp.-ext-10003/_tmp.000030_0 for DFSClient_NONMAPREDUCE-27052423_230 for client 192.168.x.x because current leaseholder is trying to recreate file.

      If we will change hive execution engine to mr and execute the same command - create table, we will get:

      Caused by: org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /user/xxxx is exceeded: quota = 10737418240 B = 10 GB but diskspace consumed = 11098812438 B = 10.34 GB

      After increasing quota hive on spark is working.
      The problem is with log message, which is inaccurate and not helpful.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Markovich Rivkin Andrey
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: