Hadoop Common
  1. Hadoop Common
  2. HADOOP-10329

Fully qualified URIs are inconsistant and sometimes break in hadoop conf files

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Component/s: conf
    • Labels:

      Description

      When specifying paths in the *-site.xml files, some are required to be fully qualified, while others (specifically hadoop.tmp.dir) break when a fully qualified uri is used.

      Example:
      If I set hadoop.tmp.dir in core-site to file:///something it'll create a file: directory in my $PWD.

        <property>
          <name>hadoop.tmp.dir</name>
          <value>file:///grid/a/tmp/hadoop-${user.name}</value>
        </property>
      
      [tthompso@test ~]$ tree file\:/
      file:/
      └── grid
          └── a
              └── tmp
                  └── hadoop-tthompso
      

      Other places, like the datanode, or the nodemanager, will complain if I don't use fully qualified uris

        <property>
          <name>dfs.datanode.data.dir</name>
          <value>/grid/a/dfs-data/bs</value>
        </property>
      
      WARN org.apache.hadoop.hdfs.server.common.Util: Path /grid/a/dfs-data/bs should be specified as a URI in configuration files. Please update hdfs configuration.
      
      1. HADOOP-10329.3.patch
        37 kB
        Mohammad Kamrul Islam
      2. HADOOP-10329.2.patch
        36 kB
        Mohammad Kamrul Islam
      3. HADOOP-10329.1.patch
        36 kB
        Mohammad Kamrul Islam

        Activity

        Allen Wittenauer made changes -
        Labels BB2015-05-TBR
        Allen Wittenauer made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Allen Wittenauer made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Mohammad Kamrul Islam made changes -
        Attachment HADOOP-10329.3.patch [ 12629258 ]
        Mohammad Kamrul Islam made changes -
        Attachment HADOOP-10329.2.patch [ 12629094 ]
        Jakob Homan made changes -
        Assignee Mohammad Kamrul Islam [ kamrul ]
        Mohammad Kamrul Islam made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Mohammad Kamrul Islam made changes -
        Field Original Value New Value
        Attachment HADOOP-10329.1.patch [ 12628919 ]
        Travis Thompson created issue -

          People

          • Assignee:
            Mohammad Kamrul Islam
            Reporter:
            Travis Thompson
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development