Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3489

EventDispatcher should have a call-back on errors for aiding tests

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      If one of the dispatched events generates an exception - the dispatcher kills the JVM via a System.exit. Unit tests end up not running - but they don't fail either.
      TestTaskAttempt is currently running like this.
      Previously - have seen TestRecovery and TestJobHistoryParsing do the same. Most of the tests would need to be looked at.

        Issue Links

          Activity

          Hide
          Vinod Kumar Vavilapalli added a comment -

          Is this jira still an issue?

          Yes, it is. But it's minor.

          Show
          Vinod Kumar Vavilapalli added a comment - Is this jira still an issue? Yes, it is. But it's minor.
          Hide
          Amol Kekre added a comment -

          Is this jira still an issue?

          Show
          Amol Kekre added a comment - Is this jira still an issue?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either.

          That is done as part of MAPREDUCE-3634 which isn't committed yet. Appreciate a quick review there

          You can develop your patch over MAPREDUCE-3634.

          Show
          Vinod Kumar Vavilapalli added a comment - Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. That is done as part of MAPREDUCE-3634 which isn't committed yet. Appreciate a quick review there You can develop your patch over MAPREDUCE-3634 .
          Hide
          Sharad Agarwal added a comment -

          Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. Is this the intended bahaviour ? I think daemons should exit on dispatcher error while testcases can handle it differently. right ?

          Show
          Sharad Agarwal added a comment - Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. Is this the intended bahaviour ? I think daemons should exit on dispatcher error while testcases can handle it differently. right ?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          let me take a stab at this.

          Great, thanks!

          Show
          Vinod Kumar Vavilapalli added a comment - let me take a stab at this. Great, thanks!
          Hide
          Sharad Agarwal added a comment -

          let me take a stab at this.

          Show
          Sharad Agarwal added a comment - let me take a stab at this.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The issue of failing silently should be fixed after MAPREDUCE-3634. There, I have changed the behavior of crashing dispatcher during tests to instead throw exceptions. We will still need some kind of a call-back so that tests can fail better. Currently all tests take care of this by explicit checks and timeouts.

          Show
          Vinod Kumar Vavilapalli added a comment - The issue of failing silently should be fixed after MAPREDUCE-3634 . There, I have changed the behavior of crashing dispatcher during tests to instead throw exceptions. We will still need some kind of a call-back so that tests can fail better. Currently all tests take care of this by explicit checks and timeouts.
          Hide
          Sharad Agarwal added a comment -

          Definately System.exit is not the right thing in a library. Sigh! I knew that when I wrote that and intended to remove it. Couldn't get chance to get to it.
          I think we should get rid of it and instead have a error handling callback registered to Dispatcher.

          Show
          Sharad Agarwal added a comment - Definately System.exit is not the right thing in a library. Sigh! I knew that when I wrote that and intended to remove it. Couldn't get chance to get to it. I think we should get rid of it and instead have a error handling callback registered to Dispatcher.

            People

            • Assignee:
              Sharad Agarwal
              Reporter:
              Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development