Hadoop Common
  1. Hadoop Common
  2. HADOOP-7685

Issues with hadoop-common-project\hadoop-common\src\main\packages\hadoop-setup-conf.sh file

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0
    • Fix Version/s: None
    • Component/s: scripts
    • Labels:
      None

      Description

      hadoop-common-project\hadoop-common\src\main\packages\hadoop-setup-conf.sh has following issues
      1. check_permission does not work as expected if there are two folders with $NAME as part of their name inside $PARENT
      e.g. /home/hadoop/conf, /home/hadoop/someconf,
      The result of `ls -ln $PARENT | grep -w $NAME| awk '

      {print $3}

      '` is non zero..it is 0 0 and hence the following if check becomes true.

      if [ "$OWNER" != "0" ]; then
      RESULT=1
      break
      fi 
      

      2. Spelling mistake

      HADDOP_DN_ADDR="0.0.0.0:50010"
      

      it should be

      HADOOP_DN_ADDR="0.0.0.0:50010"
      

      3. HADOOP_SNN_HOST is not set due to which the hdfs-site.xml contains following configuration

      <property>
      <name>dfs.namenode.http-address</name>
      <value>:50070</value>
      <description>
      The address and the base port where the dfs namenode web ui will listen on.
      If the port is 0 then the server will start on a free port.
      </description>
      </property>
      
      1. HADOOP-7685.patch
        4 kB
        Devaraj K
      2. HADOOP-7685.patch
        18 kB
        Eric Yang
      3. HADOOP-7685-0.20-security.patch
        1 kB
        Matt Foley
      4. HADOOP-7685-trunk.patch
        6 kB
        Eric Yang

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          1d 2h 21m 1 Matt Foley 28/Sep/11 20:41
          Open Open Patch Available Patch Available
          5d 5h 31m 2 Eric Yang 03/Oct/11 22:47
          Patch Available Patch Available Resolved Resolved
          4h 11m 1 Eric Yang 04/Oct/11 02:59
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178672 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1028 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1028/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1028 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1028/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1086 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1086/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1086 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1086/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1008 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1008/)
          HADOOP-7685. Improved directory ownership check function in
          hadoop-setup-conf.sh. (Eric Yang)

          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1008 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1008/ ) HADOOP-7685 . Improved directory ownership check function in hadoop-setup-conf.sh. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178671 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh
          Eric Yang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 0.23.0, 0.20.205.0 [ 12315569, 12316390 ] 0.20.205.0, 0.23.0 [ 12316390, 12315569 ]
          Resolution Fixed [ 1 ]
          Hide
          Eric Yang added a comment -

          I just committed this to trunk and 0.23, thanks Devaraj and Matt.

          Show
          Eric Yang added a comment - I just committed this to trunk and 0.23, thanks Devaraj and Matt.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496914/HADOOP-7685-0.20-security.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/258//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496914/HADOOP-7685-0.20-security.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/258//console This message is automatically generated.
          Eric Yang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.23.0, 0.20.205.0 [ 12315569, 12316390 ] 0.20.205.0, 0.23.0 [ 12316390, 12315569 ]
          Eric Yang made changes -
          Assignee Devaraj K [ devaraj.k ] Eric Yang [ eyang ]
          Hide
          Matt Foley added a comment -

          Committed to 0.20-security and 0.20.205.
          Thanks, Eric and Devaraj!

          Show
          Matt Foley added a comment - Committed to 0.20-security and 0.20.205. Thanks, Eric and Devaraj!
          Matt Foley made changes -
          Link This issue is blocked by HADOOP-7684 [ HADOOP-7684 ]
          Matt Foley made changes -
          Attachment HADOOP-7685-0.20-security.patch [ 12496914 ]
          Hide
          Matt Foley added a comment -

          The patch for 0.20-security incorrectly incorporated the patch for HADOOP-7684 also. The patch I actually applied after conflict resolution is now attached.
          I'm assuming it is actually dependent on HADOOP-7684 being applied first, so marking it as a blocking bug.

          Show
          Matt Foley added a comment - The patch for 0.20-security incorrectly incorporated the patch for HADOOP-7684 also. The patch I actually applied after conflict resolution is now attached. I'm assuming it is actually dependent on HADOOP-7684 being applied first, so marking it as a blocking bug.
          Matt Foley made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.23.0, 0.20.205.0 [ 12315569, 12316390 ] 0.20.205.0, 0.23.0 [ 12316390, 12315569 ]
          Hide
          Matt Foley added a comment -

          Note glitch in naming convention: The first "HADOOP-7685.patch" is targeted for 0.20-security, while "HADOOP-7685-trunk.patch" and the second "HADOOP-7685.patch" are targeted for trunk.

          I am committing the first "HADOOP-7685.patch" to 0.20-security and 0.20.205 now.
          I will commit the other to trunk and 0.23 shortly.

          Show
          Matt Foley added a comment - Note glitch in naming convention: The first " HADOOP-7685 .patch" is targeted for 0.20-security, while " HADOOP-7685 -trunk.patch" and the second " HADOOP-7685 .patch" are targeted for trunk. I am committing the first " HADOOP-7685 .patch" to 0.20-security and 0.20.205 now. I will commit the other to trunk and 0.23 shortly.
          Hide
          Matt Foley added a comment -

          Devaraj K verified and Eric Y reviewed. +1.
          Committing to 0.20-security and 0.20.205.
          Thanks, Devaraj and Eric!

          Show
          Matt Foley added a comment - Devaraj K verified and Eric Y reviewed. +1. Committing to 0.20-security and 0.20.205. Thanks, Devaraj and Eric!
          Matt Foley made changes -
          Affects Version/s 0.23.0 [ 12315569 ]
          Affects Version/s 0.24.0 [ 12317652 ]
          Target Version/s 0.23.0, 0.20.205.0 [ 12315569, 12316390 ] 0.20.205.0, 0.23.0 [ 12316390, 12315569 ]
          Matt Foley made changes -
          Fix Version/s 0.24.0 [ 12317652 ]
          Affects Version/s 0.20.205.0 [ 12316390 ]
          Target Version/s 0.20.205.0, 0.23.0 [ 12316390, 12315569 ]
          Hide
          Devaraj K added a comment -

          No tests needed for this patch because changes present only in script files. Verified these manually.

          Show
          Devaraj K added a comment - No tests needed for this patch because changes present only in script files. Verified these manually.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496766/HADOOP-7685.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496766/HADOOP-7685.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/237//console This message is automatically generated.
          Devaraj K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.24.0 [ 12317652 ]
          Hide
          Devaraj K added a comment -

          Updated the patch for issues listed. I have tested these manually. Please review this patch.

          Show
          Devaraj K added a comment - Updated the patch for issues listed. I have tested these manually. Please review this patch.
          Hide
          Devaraj K added a comment -

          Hi Eric, I am not sure why changes have done in 'preinst' file. If it is addressing any other problem you can file another jira.

          I am attaching a patch which addresses the problems listed above.

          Show
          Devaraj K added a comment - Hi Eric, I am not sure why changes have done in 'preinst' file. If it is addressing any other problem you can file another jira. I am attaching a patch which addresses the problems listed above.
          Devaraj K made changes -
          Attachment HADOOP-7685.patch [ 12496766 ]
          Eric Yang made changes -
          Attachment HADOOP-7685-trunk.patch [ 12496714 ]
          Hide
          Eric Yang added a comment -

          Same patch for trunk.

          Show
          Eric Yang added a comment - Same patch for trunk.
          Eric Yang made changes -
          Field Original Value New Value
          Attachment HADOOP-7685.patch [ 12496713 ]
          Hide
          Eric Yang added a comment -

          1. Improved logic in checking file ownership.
          2. Corrected spelling error.
          3. I am not sure if I am fixing this correctly, it sounds like there should be a default for HADOOP_SNN_HOST?

          Show
          Eric Yang added a comment - 1. Improved logic in checking file ownership. 2. Corrected spelling error. 3. I am not sure if I am fixing this correctly, it sounds like there should be a default for HADOOP_SNN_HOST?
          Devaraj K created issue -

            People

            • Assignee:
              Eric Yang
              Reporter:
              Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development