Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8174

Add containerId to ResourceLocalizationService fetch failure log statement

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.3
    • Fix Version/s: 3.3.0
    • Component/s: nodemanager
    • Labels:
      None

      Description

      When a localization for a resource failed due to change in timestamp, there is no containerId logged to correlate.

      2018-04-18 07:31:46,033 WARN  localizer.ResourceLocalizationService (ResourceLocalizationService.java:processHeartbeat(1017)) - { hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo, 1524036694502, FILE, null } failed: Resource hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo changed on src filesystem (expected 1524036694502, was 1524036694502
      java.io.IOException: Resource hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo changed on src filesystem (expected 1524036694502, was 1524036694502
              at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:258)
              at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63)
              at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:362)
              at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:360)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
              at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:360)
              at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      
      

        Attachments

        1. YARN-8174.3.patch
          2 kB
          Prabhu Joseph
        2. YARN-8174.2.patch
          1 kB
          Prabhu Joseph
        3. YARN-8174.1.patch
          1 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: