Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: scripts
    • Labels:
      None

      Description

      Somewhere along the way, daemons run in default mode lost pid files.

      1. HADOOP-11055.patch
        8 kB
        Allen Wittenauer
      2. HADOOP-11055-01.patch
        8 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          OK, this appears to be fairly easy to solve. The big thing is that the pid file var needs to get sent to start_daemon so that it can write the pid file before launching java. The exec will replace the running process with the java, but the pid should not change.

          This, however, means breaking/updating the params that get sent to start_daemon. Since this is an unreleased version, not a big deal.

          Show
          aw Allen Wittenauer added a comment - OK, this appears to be fairly easy to solve. The big thing is that the pid file var needs to get sent to start_daemon so that it can write the pid file before launching java. The exec will replace the running process with the java, but the pid should not change. This, however, means breaking/updating the params that get sent to start_daemon. Since this is an unreleased version, not a big deal.
          Hide
          aw Allen Wittenauer added a comment -

          It's worth noting that _wrapper should keep its pid code since we are forking the java bit. This will replace that pid with the forked one.

          Show
          aw Allen Wittenauer added a comment - It's worth noting that _wrapper should keep its pid code since we are forking the java bit. This will replace that pid with the forked one.
          Hide
          pocky John Smith added a comment -

          Isn't it safe to write the pid file inside the hadoop_start_daemon function since that is the process id of the pid file?

          Show
          pocky John Smith added a comment - Isn't it safe to write the pid file inside the hadoop_start_daemon function since that is the process id of the pid file?
          Hide
          aw Allen Wittenauer added a comment -

          Yup, you are absolutely correct. There's no point in keeping the pid write after the fork since it should write the exact same info anyway.

          Show
          aw Allen Wittenauer added a comment - Yup, you are absolutely correct. There's no point in keeping the pid write after the fork since it should write the exact same info anyway.
          Hide
          aw Allen Wittenauer added a comment -
          • Fixes the error in secure_prereq to resemble English
          • _daemon functions now write the pidfile on execution
          • _wrapper functions now overwrite the pidfile with the proper post-fork pid
          • Counter put in place to avoid a race condition with pid file creation vs. fork pid
          • Cleaned up daemon related error messages
          • Fixed disown to actually operate on the correct pid
          Show
          aw Allen Wittenauer added a comment - Fixes the error in secure_prereq to resemble English _daemon functions now write the pidfile on execution _wrapper functions now overwrite the pidfile with the proper post-fork pid Counter put in place to avoid a race condition with pid file creation vs. fork pid Cleaned up daemon related error messages Fixed disown to actually operate on the correct pid
          Hide
          aw Allen Wittenauer added a comment -

          Playing around, the $$ inside the &'d process is wrong for the daemonization case during the pid write. So John Smith, we were both wrong after some experimentation!

          Show
          aw Allen Wittenauer added a comment - Playing around, the $$ inside the &'d process is wrong for the daemonization case during the pid write. So John Smith , we were both wrong after some experimentation!
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12668452/HADOOP-11055.patch
          against trunk revision 54e5794.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4706//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4706//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12668452/HADOOP-11055.patch against trunk revision 54e5794. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4706//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4706//console This message is automatically generated.
          Hide
          owen.omalley Owen O'Malley added a comment -

          +1 looks good, Allen.

          Show
          owen.omalley Owen O'Malley added a comment - +1 looks good, Allen.
          Hide
          aw Allen Wittenauer added a comment -

          -01 is just some syntax cleanup.

          Thanks Owen! I'll commit this in a bit.

          Show
          aw Allen Wittenauer added a comment - -01 is just some syntax cleanup. Thanks Owen! I'll commit this in a bit.
          Hide
          aw Allen Wittenauer added a comment -

          Committed to trunk.

          Show
          aw Allen Wittenauer added a comment - Committed to trunk.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12669202/HADOOP-11055-01.patch
          against trunk revision 56119fe.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The following test timeouts occurred in hadoop-common-project/hadoop-common:

          org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4739//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4739//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12669202/HADOOP-11055-01.patch against trunk revision 56119fe. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The following test timeouts occurred in hadoop-common-project/hadoop-common: org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4739//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4739//console This message is automatically generated.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/683/)
          HADOOP-11055. non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/683/ ) HADOOP-11055 . non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1899 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1899/)
          HADOOP-11055. non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1899 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1899/ ) HADOOP-11055 . non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1874 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1874/)
          HADOOP-11055. non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1874 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1874/ ) HADOOP-11055 . non-daemon pid files are missing (aw) (aw: rev ffdb7eb3b256c8e064948a962aa07e2e276889c5) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development