Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-351

multilang python process fall into endless loop

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.9.3
    • Fix Version/s: 0.9.3
    • Component/s: storm-multilang
    • Labels:
      None
    • Environment:
      storm 0.9.3-incubating

      Description

      1. steps to reproduce
      1) write a topology with a python bolt, run the topology on storm; then there will be two process for the bolt: the worker(java process for ShellBolt), python process.
      2)kill -9 the worker(java process for ShellBolt);

      2. expected behavior
      the worker exit and the python process exist

      3. actual, incorrect behavior
      the worker exit, but the python process never exist and fall into endless loop

      4. analyse
      in storm.py,read tuple from stdin with follow function:

      def readMsg():
      msg = ""
      while True:
      line = sys.stdin.readline()[0:-1]
      if line == "end":
      break
      msg = msg + line + "\n"
      return json_decode(msg[0:-1])

      when sys.stdin is closed, EOF is encountered, readline() return None, so readMsg fall into endless loop.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mazhechao commented on the pull request:

          https://github.com/apache/storm/pull/140#issuecomment-147574031

          I mean can this bug lead to a memory leak.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mazhechao commented on the pull request: https://github.com/apache/storm/pull/140#issuecomment-147574031 I mean can this bug lead to a memory leak.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user revans2 commented on the pull request:

          https://github.com/apache/storm/pull/140#issuecomment-147405836

          @mazhechao are you asking if the fix can lead to a memory leak? I'm not really sure what you mean by this.

          Show
          githubbot ASF GitHub Bot added a comment - Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/140#issuecomment-147405836 @mazhechao are you asking if the fix can lead to a memory leak? I'm not really sure what you mean by this.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mazhechao commented on the pull request:

          https://github.com/apache/storm/pull/140#issuecomment-147291501

          Can this leads to memory leak?

          Show
          githubbot ASF GitHub Bot added a comment - Github user mazhechao commented on the pull request: https://github.com/apache/storm/pull/140#issuecomment-147291501 Can this leads to memory leak?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-storm/pull/140

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-storm/pull/140
          Hide
          dagit Derek Dagit added a comment -

          Thanks DashengJu, I merged this into master.

          Show
          dagit Derek Dagit added a comment - Thanks DashengJu, I merged this into master.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user d2r commented on the pull request:

          https://github.com/apache/incubator-storm/pull/140#issuecomment-48600374

          Looks OK to me. +1

          Show
          githubbot ASF GitHub Bot added a comment - Github user d2r commented on the pull request: https://github.com/apache/incubator-storm/pull/140#issuecomment-48600374 Looks OK to me. +1
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dashengju commented on the pull request:

          https://github.com/apache/incubator-storm/pull/140#issuecomment-48563686

          @d2r, @ptgoetz, please help to review this pull request, thank you very much

          Show
          githubbot ASF GitHub Bot added a comment - Github user dashengju commented on the pull request: https://github.com/apache/incubator-storm/pull/140#issuecomment-48563686 @d2r, @ptgoetz, please help to review this pull request, thank you very much
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dashengju commented on the pull request:

          https://github.com/apache/incubator-storm/pull/140#issuecomment-46259448

          I have changed the title, thank you

          Show
          githubbot ASF GitHub Bot added a comment - Github user dashengju commented on the pull request: https://github.com/apache/incubator-storm/pull/140#issuecomment-46259448 I have changed the title, thank you
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user ptgoetz opened a pull request:

          https://github.com/apache/incubator-storm/pull/141

          STORM-351: fix logic for writing offsets to ZooKeeper

          Should be self-explanatory from the diff.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/ptgoetz/incubator-storm STORM-351

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-storm/pull/141.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #141


          commit 6f9a67385cf5418b7ae9c6b1896dc573c5ec8a3b
          Author: P. Taylor Goetz <ptgoetz@gmail.com>
          Date: 2014-06-13T18:01:32Z

          STORM-351: fix logic for writing offsets to ZooKeeper


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user ptgoetz opened a pull request: https://github.com/apache/incubator-storm/pull/141 STORM-351 : fix logic for writing offsets to ZooKeeper Should be self-explanatory from the diff. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ptgoetz/incubator-storm STORM-351 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-storm/pull/141.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #141 commit 6f9a67385cf5418b7ae9c6b1896dc573c5ec8a3b Author: P. Taylor Goetz <ptgoetz@gmail.com> Date: 2014-06-13T18:01:32Z STORM-351 : fix logic for writing offsets to ZooKeeper

            People

            • Assignee:
              dashengju DashengJu
              Reporter:
              dashengju DashengJu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m

                  Development