Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6847

Job history server should release jobs from cache after a fixed duration

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: jobhistoryserver
    • Labels:
      None

      Description

      We found history server is consuming a lot of memory when there are large jobs (with more than 100k of tasks in a single job). Currently JHS cache only evicts entries with size, it's better to add the time expiration as well to reduce heap usage if job has no one accessing for sometime.

        Attachments

        1. MAPREDUCE-6847.01.patch
          7 kB
          Weiwei Yang

          Activity

            People

            • Assignee:
              cheersyang Weiwei Yang
              Reporter:
              cheersyang Weiwei Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: