Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3558

Additional containers getting reserved from RM in case of Fair scheduler

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.0
    • None
    • None
    • OS :Suse 11 Sp3
      Setup : 2 RM 2 NM
      Scheduler : Fair scheduler

    Description

      Submit PI job with 16 maps
      Total container expected : 16 MAPS + 1 Reduce + 1 AM
      Total containers reserved by RM is 21

      Below set of containers are not being used for execution

      container_1430213948957_0001_01_000020
      container_1430213948957_0001_01_000019

      RM Containers reservation and states

       Processing container_1430213948957_0001_01_000001 of type START
       Processing container_1430213948957_0001_01_000001 of type ACQUIRED
       Processing container_1430213948957_0001_01_000001 of type LAUNCHED
       Processing container_1430213948957_0001_01_000002 of type START
       Processing container_1430213948957_0001_01_000003 of type START
       Processing container_1430213948957_0001_01_000002 of type ACQUIRED
       Processing container_1430213948957_0001_01_000003 of type ACQUIRED
       Processing container_1430213948957_0001_01_000004 of type START
       Processing container_1430213948957_0001_01_000005 of type START
       Processing container_1430213948957_0001_01_000004 of type ACQUIRED
       Processing container_1430213948957_0001_01_000005 of type ACQUIRED
       Processing container_1430213948957_0001_01_000002 of type LAUNCHED
       Processing container_1430213948957_0001_01_000004 of type LAUNCHED
       Processing container_1430213948957_0001_01_000006 of type RESERVED
       Processing container_1430213948957_0001_01_000003 of type LAUNCHED
       Processing container_1430213948957_0001_01_000005 of type LAUNCHED
       Processing container_1430213948957_0001_01_000007 of type START
       Processing container_1430213948957_0001_01_000007 of type ACQUIRED
       Processing container_1430213948957_0001_01_000007 of type LAUNCHED
       Processing container_1430213948957_0001_01_000008 of type RESERVED
       Processing container_1430213948957_0001_01_000002 of type FINISHED
       Processing container_1430213948957_0001_01_000006 of type START
       Processing container_1430213948957_0001_01_000006 of type ACQUIRED
       Processing container_1430213948957_0001_01_000006 of type LAUNCHED
       Processing container_1430213948957_0001_01_000004 of type FINISHED
       Processing container_1430213948957_0001_01_000009 of type START
       Processing container_1430213948957_0001_01_000009 of type ACQUIRED
       Processing container_1430213948957_0001_01_000009 of type LAUNCHED
       Processing container_1430213948957_0001_01_000010 of type RESERVED
       Processing container_1430213948957_0001_01_000003 of type FINISHED
       Processing container_1430213948957_0001_01_000008 of type START
       Processing container_1430213948957_0001_01_000008 of type ACQUIRED
       Processing container_1430213948957_0001_01_000008 of type LAUNCHED
       Processing container_1430213948957_0001_01_000005 of type FINISHED
       Processing container_1430213948957_0001_01_000011 of type START
       Processing container_1430213948957_0001_01_000011 of type ACQUIRED
       Processing container_1430213948957_0001_01_000011 of type LAUNCHED
       Processing container_1430213948957_0001_01_000007 of type FINISHED
       Processing container_1430213948957_0001_01_000012 of type START
       Processing container_1430213948957_0001_01_000012 of type ACQUIRED
       Processing container_1430213948957_0001_01_000012 of type LAUNCHED
       Processing container_1430213948957_0001_01_000013 of type RESERVED
       Processing container_1430213948957_0001_01_000006 of type FINISHED
       Processing container_1430213948957_0001_01_000010 of type START
       Processing container_1430213948957_0001_01_000010 of type ACQUIRED
       Processing container_1430213948957_0001_01_000010 of type LAUNCHED
       Processing container_1430213948957_0001_01_000009 of type FINISHED
       Processing container_1430213948957_0001_01_000014 of type START
       Processing container_1430213948957_0001_01_000014 of type ACQUIRED
       Processing container_1430213948957_0001_01_000014 of type LAUNCHED
       Processing container_1430213948957_0001_01_000015 of type RESERVED
       Processing container_1430213948957_0001_01_000008 of type FINISHED
       Processing container_1430213948957_0001_01_000013 of type START
       Processing container_1430213948957_0001_01_000016 of type RESERVED
       Processing container_1430213948957_0001_01_000013 of type ACQUIRED
       Processing container_1430213948957_0001_01_000013 of type LAUNCHED
       Processing container_1430213948957_0001_01_000011 of type FINISHED
       Processing container_1430213948957_0001_01_000016 of type START
       Processing container_1430213948957_0001_01_000010 of type FINISHED
       Processing container_1430213948957_0001_01_000015 of type START
       Processing container_1430213948957_0001_01_000016 of type ACQUIRED
       Processing container_1430213948957_0001_01_000015 of type ACQUIRED
       Processing container_1430213948957_0001_01_000016 of type LAUNCHED
       Processing container_1430213948957_0001_01_000012 of type FINISHED
       Processing container_1430213948957_0001_01_000017 of type START
       Processing container_1430213948957_0001_01_000015 of type LAUNCHED
       Processing container_1430213948957_0001_01_000014 of type FINISHED
       Processing container_1430213948957_0001_01_000018 of type START
       Processing container_1430213948957_0001_01_000017 of type ACQUIRED
       Processing container_1430213948957_0001_01_000018 of type ACQUIRED
       Processing container_1430213948957_0001_01_000017 of type LAUNCHED
       Processing container_1430213948957_0001_01_000019 of type RESERVED
       Processing container_1430213948957_0001_01_000020 of type RESERVED
       Processing container_1430213948957_0001_01_000018 of type RELEASED
       Processing container_1430213948957_0001_01_000015 of type FINISHED
       Processing container_1430213948957_0001_01_000013 of type FINISHED
       Processing container_1430213948957_0001_01_000016 of type FINISHED
       Processing container_1430213948957_0001_01_000017 of type FINISHED
       Processing container_1430213948957_0001_01_000021 of type START
       Processing container_1430213948957_0001_01_000021 of type ACQUIRED
       Processing container_1430213948957_0001_01_000021 of type LAUNCHED
       Processing container_1430213948957_0001_01_000021 of type FINISHED
       Processing container_1430213948957_0001_01_000001 of type FINISHED
      

      AM assignment to Nodes

      ---------- Find in open document(s) ----------
      Assigning container container_1430213948957_0001_01_000002 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000003 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000004 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000005 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000007 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000006 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000009 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000008 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000011 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000012 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000010 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000014 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000013 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000016 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000015 with priority 20 to NM <NODE1>:64318
      Assigning container container_1430213948957_0001_01_000017 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000018 with priority 20 to NM <NODE2>:64318
      Assigning container container_1430213948957_0001_01_000021 with priority 10 to NM <NODE2>:64318
      18 occurrences have been found.
      Output completed (0 sec consumed)
      

      Attachments

        1. rm.log
          1.90 MB
          Bibin Chundatt
        2. Amlog.txt
          1.35 MB
          Bibin Chundatt

        Activity

          People

            sunilg Sunil G
            bibinchundatt Bibin Chundatt
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: