Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The JobHistory server's locking is inconsistent and wrong in some cases. This is not super critical because the issues would only show up if a job is being cleaned up or moved from intermediate done to done, at the same time it is being parsed into a CompletedJob. However the locking is slowing down the server in some cases, and is a ticking time bomb that needs to be addressed.

      As part of this too we need to be sure that the Cleaner and Intermediate to Done migration threads handle exceptions properly. Now it appears that the exception is logged, and the thread just shuts down. This means that the history server could still be up and running for weeks and never remove old jobs.

      1. MR-3972.txt
        80 kB
        Robert Joseph Evans
      2. MR-3972.txt
        75 kB
        Robert Joseph Evans
      3. MR-3972.txt
        51 kB
        Robert Joseph Evans
      4. MR-3972.txt
        49 kB
        Robert Joseph Evans
      5. MR-3972.txt
        50 kB
        Robert Joseph Evans
      6. MR-3972.txt
        50 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322471 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.23.3 [ 12320060 ]
          Resolution Fixed [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12522994 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12522611 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12522266 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12522117 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12521221 ]
          Robert Joseph Evans made changes -
          Link This issue is duplicated by MAPREDUCE-4080 [ MAPREDUCE-4080 ]
          Robert Joseph Evans made changes -
          Attachment MR-3972.txt [ 12520302 ]
          Robert Joseph Evans made changes -
          Link This issue is related to MAPREDUCE-4055 [ MAPREDUCE-4055 ]
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Parent MAPREDUCE-3973 [ 12545239 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]
          Robert Joseph Evans created issue -

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development