Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4103

Progress in DAG, Vertex, and tasks is incorrect

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.0, 0.9.3
    • None
    • None

    Description

      Looking at the progress code, there some few issues that could lead to some problems calculating the progress.
      There are some cases when the progress never reach 1.0.
      This is a list of issues that need to be fixed in the progress code:

      • After TEZ-3982, since values are skipped in the In some cases, the progress of DAG or a vertex may never reach 1.0f. this is in both "DAGImpl.java" and "ProgressHelper.java"
      • ProgressHelper schedules a service to update the progress, dubbed `ProgressHelper.monitorProgress`. According to Java Documentation:

        If any execution of the task encounters an exception,
        subsequent executions are suppressed.
        Otherwise, the task will only terminate via cancellation
        or termination of the executor.

        In other words, if the service dies, there is no way to catch that in the code and the progress will never be updated.

      • The `SimpleProcessor.inputMap` is not thread-safe. They are initialized as `LinkedHashMap` and there is no synchronization on the field objects in the map. This could be problematic in concurrent context.
      • `VertexImpl.getProgress()` does not check the range of the progress calculated in `VertexImpl.computeProgress()`
         

      Attachments

        1. TEZ-4103.006.patch
          19 kB
          Ahmed Hussein
        2. TEZ-4103.005.patch
          19 kB
          Ahmed Hussein
        3. TEZ-4103.004.patch
          14 kB
          Ahmed Hussein
        4. TEZ-4103.003.patch
          16 kB
          Ahmed Hussein
        5. TEZ-4103.002.patch
          15 kB
          Ahmed Hussein
        6. TEZ-4103.001.patch
          14 kB
          Ahmed Hussein

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ahussein Ahmed Hussein
            ahussein Ahmed Hussein
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment