Details

    • Hadoop Flags:
      Reviewed

      Description

      lWhen the JHS starts up, it initializes the internal caches and storage via the HistoryFileManager. If we have a large number of existing finished jobs then we could spent minutes in this startup phase without logging progress:
      2016-03-14 10:56:01,444 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://hadoopcdh.itnas01.ieee.org:8020]
      2016-03-14 10:56:11,455 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing Existing Jobs...
      2016-03-14 12:01:36,926 INFO org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage: CachedHistoryStorage Init
      This makes it really difficult to assess if things are working correctly (it looks hung). We can add logs to notify users of progress.

        Attachments

        1. mapreduce6718.001.patch
          2 kB
          Haibo Chen
        2. mapreduce6718.002.patch
          2 kB
          Haibo Chen
        3. mapreduce6718.003.patch
          3 kB
          Haibo Chen

          Activity

            People

            • Assignee:
              haibochen Haibo Chen
              Reporter:
              haibochen Haibo Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: