Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2808

yarn client tool can not list app_attempt's container info correctly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When enabling timeline server, yarn client can not list the container info for a application attempt correctly.
      Here is the reproduce step.

      1. enabling yarn timeline server
      2. submit a MR job
      3. after the job is finished. use yarn client to list the container info of the app attempt.

      Then, since the RM has cached the application's attempt info, the output show

      [hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list appattempt_1415168250217_0001_000001
      14/11/05 01:19:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      14/11/05 01:19:15 INFO impl.TimelineClientImpl: Timeline service address: http://0.0.0.0:8188/ws/v1/timeline/
      14/11/05 01:19:16 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
      14/11/05 01:19:16 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200
      Total number of containers :0
                        Container-Id	          Start Time	         Finish Time	               State	                Host	                            LOG-URL
      

      But if the rm is restarted, client can fetch the container info from timeline server correctly.

      [hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list appattempt_1415168250217_0001_000001
      14/11/05 01:21:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      14/11/05 01:21:06 INFO impl.TimelineClientImpl: Timeline service address: http://0.0.0.0:8188/ws/v1/timeline/
      14/11/05 01:21:06 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
      14/11/05 01:21:06 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200
      Total number of containers :4
                        Container-Id	          Start Time	         Finish Time	               State	                Host	                            LOG-URL
      container_1415168250217_0001_01_000001	       1415168318376	       1415168349896	            COMPLETE	localhost.localdomain:47024	http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000001/container_1415168250217_0001_01_000001/hadoop
      container_1415168250217_0001_01_000002	       1415168326399	       1415168334858	            COMPLETE	localhost.localdomain:47024	http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000002/container_1415168250217_0001_01_000002/hadoop
      container_1415168250217_0001_01_000003	       1415168326400	       1415168335277	            COMPLETE	localhost.localdomain:47024	http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000003/container_1415168250217_0001_01_000003/hadoop
      container_1415168250217_0001_01_000004	       1415168335825	       1415168343873	            COMPLETE	localhost.localdomain:47024	http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000004/container_1415168250217_0001_01_000004/hadoop
      

        Attachments

        1. YARN-2808.20150131-1.patch
          11 kB
          Naganarasimha G R
        2. YARN-2808.20150130-1.patch
          9 kB
          Naganarasimha G R
        3. YARN-2808.20150126-1.patch
          8 kB
          Naganarasimha G R

          Issue Links

            Activity

              People

              • Assignee:
                Naganarasimha Naganarasimha G R
                Reporter:
                wangg23 Gordon Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: