Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16167

"hadoop CLASSFILE" prints error messages on Ubuntu 18

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.3.0
    • scripts
    • None

    Description

      # hadoop org.apache.hadoop.conf.Configuration
      /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
      /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331: HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
      /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426: HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
      

      The issue is a regression in bash 4.4. See here. The extraneous output can break scripts that read the command output.

      According to aw:

      Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn, hdfs, etc) just needs some special handling when a custom method is being called. For example, there’s no point in checking to see if it should run with privileges, so just skip over that. Probably a few other places too. Relatively easy fix. 2 lines of code, maybe.

      Attachments

        1. HADOOP-16167.004.patch
          1 kB
          Daniel Templeton
        2. HADOOP-16167.003.patch
          1 kB
          Daniel Templeton
        3. HADOOP-16167.002.patch
          2 kB
          Daniel Templeton
        4. HADOOP-16167.001.patch
          3 kB
          Daniel Templeton

        Activity

          People

            templedf Daniel Templeton
            templedf Daniel Templeton
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: