Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6070

yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • 2.4.0, 2.5.0
    • 2.6.0
    • documentation
    • None
    • Reviewed

    Description

      We should document the condition when uber mode is enabled. Currently, users need to read following code to understand the condition.

          boolean smallMemory =
              ( (Math.max(conf.getLong(MRJobConfig.MAP_MEMORY_MB, 0),
                  conf.getLong(MRJobConfig.REDUCE_MEMORY_MB, 0))
                  <= sysMemSizeForUberSlot)
                  || (sysMemSizeForUberSlot == JobConf.DISABLED_MEMORY_LIMIT));
         boolean smallCpu =
                  Math.max(
                      conf.getInt(
                          MRJobConfig.MAP_CPU_VCORES, 
                          MRJobConfig.DEFAULT_MAP_CPU_VCORES), 
                      conf.getInt(
                          MRJobConfig.REDUCE_CPU_VCORES, 
                          MRJobConfig.DEFAULT_REDUCE_CPU_VCORES)) 
                   <= sysCPUSizeForUberSlot
      

      Attachments

        1. YARN-2078.1.patch
          1 kB
          Tsuyoshi Ozawa
        2. MAPREDUCE-6070.2.patch
          1 kB
          Tsuyoshi Ozawa

        Activity

          People

            ozawa Tsuyoshi Ozawa
            ozawa Tsuyoshi Ozawa
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: