Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.21.0
    • Fix Version/s: None
    • Component/s: jobtracker
    • Labels:

      Description

      Too many tasks will eat up a considerable amount of JobTracker's heap space. According to our observation, 50GB heap size can support to 5,000,000 tasks, so we should optimize jobtracker's memory usage for more jobs and tasks. Yourkit java profile show that counters, duplicate strings, task waste too much memory. Our optimization around these three points reduced jobtracker's memory to 1/3.

        Activity

        MengWang created issue -
        MengWang made changes -
        Field Original Value New Value
        Attachment jt-memory-useage.bmp [ 12471907 ]
        MengWang made changes -
        Description To many tasks will eat up a considerable amount of JobTracker's heap space. According to our observation, 50GB heap size can support to 5,000,000 tasks, so we should optimize jobtracker's memory usage for more jobs and tasks. Yourkit java profile show that counters, duplicate strings, Task waste too much memory. Our optimization around these three points reduced jobtracker's memory to 1/3. Too many tasks will eat up a considerable amount of JobTracker's heap space. According to our observation, 50GB heap size can support to 5,000,000 tasks, so we should optimize jobtracker's memory usage for more jobs and tasks. Yourkit java profile show that counters, duplicate strings, task waste too much memory. Our optimization around these three points reduced jobtracker's memory to 1/3.
        Arun C Murthy made changes -
        Fix Version/s 0.24.0 [ 12317654 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Allen Wittenauer made changes -
        Fix Version/s 0.24.0 [ 12317654 ]
        Allen Wittenauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            MengWang
          • Votes:
            1 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development