Hadoop Common
  1. Hadoop Common
  2. HADOOP-7685

Issues with hadoop-common-project\hadoop-common\src\main\packages\hadoop-setup-conf.sh file

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0
    • Fix Version/s: None
    • Component/s: scripts
    • Labels:
      None

      Description

      hadoop-common-project\hadoop-common\src\main\packages\hadoop-setup-conf.sh has following issues
      1. check_permission does not work as expected if there are two folders with $NAME as part of their name inside $PARENT
      e.g. /home/hadoop/conf, /home/hadoop/someconf,
      The result of `ls -ln $PARENT | grep -w $NAME| awk '

      {print $3}

      '` is non zero..it is 0 0 and hence the following if check becomes true.

      if [ "$OWNER" != "0" ]; then
      RESULT=1
      break
      fi 
      

      2. Spelling mistake

      HADDOP_DN_ADDR="0.0.0.0:50010"
      

      it should be

      HADOOP_DN_ADDR="0.0.0.0:50010"
      

      3. HADOOP_SNN_HOST is not set due to which the hdfs-site.xml contains following configuration

      <property>
      <name>dfs.namenode.http-address</name>
      <value>:50070</value>
      <description>
      The address and the base port where the dfs namenode web ui will listen on.
      If the port is 0 then the server will start on a free port.
      </description>
      </property>
      
      1. HADOOP-7685-0.20-security.patch
        1 kB
        Matt Foley
      2. HADOOP-7685.patch
        4 kB
        Devaraj K
      3. HADOOP-7685-trunk.patch
        6 kB
        Eric Yang
      4. HADOOP-7685.patch
        18 kB
        Eric Yang

        Issue Links

          Activity

          Hide
          Eric Yang added a comment -

          1. Improved logic in checking file ownership.
          2. Corrected spelling error.
          3. I am not sure if I am fixing this correctly, it sounds like there should be a default for HADOOP_SNN_HOST?

          Show
          Eric Yang added a comment - 1. Improved logic in checking file ownership. 2. Corrected spelling error. 3. I am not sure if I am fixing this correctly, it sounds like there should be a default for HADOOP_SNN_HOST?
          Hide
          Eric Yang added a comment -

          Same patch for trunk.

          Show
          Eric Yang added a comment - Same patch for trunk.
          Hide
          Devaraj K added a comment -

          Hi Eric, I am not sure why changes have done in 'preinst' file. If it is addressing any other problem you can file another jira.

          I am attaching a patch which addresses the problems listed above.

          Show
          Devaraj K added a comment - Hi Eric, I am not sure why changes have done in 'preinst' file. If it is addressing any other problem you can file another jira. I am attaching a patch which addresses the problems listed above.
          Hide
          Devaraj K added a comment -

          Updated the patch for issues listed. I have tested these manually. Please review this patch.

          Show
          Devaraj K added a comment - Updated the patch for issues listed. I have tested these manually. Please review this patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496766/HADOOP-7685.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496766/HADOOP-7685.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//console This message is automatically generated.
          Hide
          Devaraj K added a comment -

          No tests needed for this patch because changes present only in script files. Verified these manually.

          Show
          Devaraj K added a comment - No tests needed for this patch because changes present only in script files. Verified these manually.
          Hide
          Matt Foley added a comment -

          Devaraj K verified and Eric Y reviewed. +1.
          Committing to 0.20-security and 0.20.205.
          Thanks, Devaraj and Eric!

          Show
          Matt Foley added a comment - Devaraj K verified and Eric Y reviewed. +1. Committing to 0.20-security and 0.20.205. Thanks, Devaraj and Eric!
          Hide
          Matt Foley added a comment -

          Note glitch in naming convention: The first "HADOOP-7685.patch" is targeted for 0.20-security, while "HADOOP-7685-trunk.patch" and the second "HADOOP-7685.patch" are targeted for trunk.

          I am committing the first "HADOOP-7685.patch" to 0.20-security and 0.20.205 now.
          I will commit the other to trunk and 0.23 shortly.

          Show
          Matt Foley added a comment - Note glitch in naming convention: The first " HADOOP-7685 .patch" is targeted for 0.20-security, while " HADOOP-7685 -trunk.patch" and the second " HADOOP-7685 .patch" are targeted for trunk. I am committing the first " HADOOP-7685 .patch" to 0.20-security and 0.20.205 now. I will commit the other to trunk and 0.23 shortly.
          Hide
          Matt Foley added a comment -

          The patch for 0.20-security incorrectly incorporated the patch for HADOOP-7684 also. The patch I actually applied after conflict resolution is now attached.
          I'm assuming it is actually dependent on HADOOP-7684 being applied first, so marking it as a blocking bug.

          Show
          Matt Foley added a comment - The patch for 0.20-security incorrectly incorporated the patch for HADOOP-7684 also. The patch I actually applied after conflict resolution is now attached. I'm assuming it is actually dependent on HADOOP-7684 being applied first, so marking it as a blocking bug.
          Hide
          Matt Foley added a comment -

          Committed to 0.20-security and 0.20.205.
          Thanks, Eric and Devaraj!

          Show
          Matt Foley added a comment - Committed to 0.20-security and 0.20.205. Thanks, Eric and Devaraj!
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496914/HADOOP-7685-0.20-security.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/258//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496914/HADOOP-7685-0.20-security.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/258//console This message is automatically generated.
          Hide
          Eric Yang added a comment -

          I just committed this to trunk and 0.23, thanks Devaraj and Matt.

          Show
          Eric Yang added a comment - I just committed this to trunk and 0.23, thanks Devaraj and Matt.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1008 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1008/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1008 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1008/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1086 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1086/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1086 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1086/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1028 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1028/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1028 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1028/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh

            People

            • Assignee:
              Eric Yang
              Reporter:
              Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development