Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8174

Add containerId to ResourceLocalizationService fetch failure log statement

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.7.3
    • 3.3.0
    • nodemanager
    • None

    Description

      When a localization for a resource failed due to change in timestamp, there is no containerId logged to correlate.

      2018-04-18 07:31:46,033 WARN  localizer.ResourceLocalizationService (ResourceLocalizationService.java:processHeartbeat(1017)) - { hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo, 1524036694502, FILE, null } failed: Resource hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo changed on src filesystem (expected 1524036694502, was 1524036694502
      java.io.IOException: Resource hdfs://tarunhdp-1.openstacklocal:8020/user/ambari-qa/.staging/job_1523550428406_0016/job.splitmetainfo changed on src filesystem (expected 1524036694502, was 1524036694502
              at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:258)
              at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63)
              at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:362)
              at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:360)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
              at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:360)
              at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      
      

      Attachments

        1. YARN-8174.1.patch
          1 kB
          Prabhu Joseph
        2. YARN-8174.2.patch
          1 kB
          Prabhu Joseph
        3. YARN-8174.3.patch
          2 kB
          Prabhu Joseph

        Activity

          People

            prabhujoseph Prabhu Joseph
            prabhujoseph Prabhu Joseph
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: