Hive
  1. Hive
  2. HIVE-6636

/user/hive is a bad default for HDFS jars path for Tez

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hive on Tez will now use /user/<current Hadoop user name>, rather than just /user/hive, as default HDFS directory for jars. {{hive.jar.directory}} can still be used to set the path explicitly.

      Description

      If user runs hive under the user name that is not "hive", jobs will fail until everyone is granted write access to /user/hive, which is not nice.

      1. HIVE-6636.patch
        2 kB
        Sergey Shelukhin
      2. HIVE-6636.02.patch
        3 kB
        Sergey Shelukhin
      3. HIVE-6636.01.patch
        3 kB
        Sergey Shelukhin

        Issue Links

          Activity

          Sergey Shelukhin created issue -
          Sergey Shelukhin made changes -
          Field Original Value New Value
          Attachment HIVE-6636.patch [ 12634212 ]
          Sergey Shelukhin made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Hive on Tez will now use /user/<current Hadoop user name>, rather than just /user/hive, as default HDFS directory for jars. {{hive.jar.directory}} can still be used to set the path explicitly.
          Fix Version/s 0.13.0 [ 12324986 ]
          Hide
          Gunther Hagleitner added a comment -

          Looks good except:

          • You're setting the var to "", I think that means that your null check won't catch that the var wasn't set and you really catch it only in the fs.status call. Why not set it to null in the first place?
          • You should update the hive-default.xml.template too
          Show
          Gunther Hagleitner added a comment - Looks good except: You're setting the var to "", I think that means that your null check won't catch that the var wasn't set and you really catch it only in the fs.status call. Why not set it to null in the first place? You should update the hive-default.xml.template too
          Hide
          Sergey Shelukhin added a comment -

          1 - last-minute change, will fix; 2 - will do

          Show
          Sergey Shelukhin added a comment - 1 - last-minute change, will fix; 2 - will do
          Hide
          Sergey Shelukhin added a comment -

          1 actually I'm passing the default explicitly:
          + String hdfsDirPathStr = HiveConf.getVar(conf, HiveConf.ConfVars.HIVE_JAR_DIRECTORY, null);

          Show
          Sergey Shelukhin added a comment - 1 actually I'm passing the default explicitly: + String hdfsDirPathStr = HiveConf.getVar(conf, HiveConf.ConfVars.HIVE_JAR_DIRECTORY, null);
          Sergey Shelukhin made changes -
          Attachment HIVE-6636.01.patch [ 12634219 ]
          Hide
          Gunther Hagleitner added a comment -

          Passing the default means that if the conf doesn't have it return null. But you set it explicitely in HiveConf. So that will never trigger. At least that's my understanding.

          Show
          Gunther Hagleitner added a comment - Passing the default means that if the conf doesn't have it return null. But you set it explicitely in HiveConf. So that will never trigger. At least that's my understanding.
          Hide
          Sergey Shelukhin added a comment -

          i think it works the other way around, overriding the default default; anyway I can change it

          Show
          Sergey Shelukhin added a comment - i think it works the other way around, overriding the default default; anyway I can change it
          Sergey Shelukhin made changes -
          Attachment HIVE-6636.02.patch [ 12634245 ]
          Hide
          Gunther Hagleitner added a comment -

          ugh. you're right. it does replace the default value. +1

          Show
          Gunther Hagleitner added a comment - ugh. you're right. it does replace the default value. +1
          Hide
          Sergey Shelukhin added a comment -

          committed to trunk and 0.13 branch

          Show
          Sergey Shelukhin added a comment - committed to trunk and 0.13 branch
          Sergey Shelukhin made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Lefty Leverenz made changes -
          Link This issue relates to HIVE-5003 [ HIVE-5003 ]
          Lefty Leverenz made changes -
          Link This issue relates to HIVE-6172 [ HIVE-6172 ]
          Lefty Leverenz made changes -
          Link This issue relates to HIVE-6098 [ HIVE-6098 ]
          Hide
          Lefty Leverenz added a comment -

          Documented in the wiki:

          Show
          Lefty Leverenz added a comment - Documented in the wiki: hive.jar.directory
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          12m 18s 1 Sergey Shelukhin 12/Mar/14 18:05
          Patch Available Patch Available Resolved Resolved
          1d 23h 56m 1 Sergey Shelukhin 14/Mar/14 18:01

            People

            • Assignee:
              Sergey Shelukhin
              Reporter:
              Sergey Shelukhin
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development