Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4080

FileNotFoundException while accessing job configuration from UI.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: jobhistoryserver, mrv2
    • Labels:
      None

      Description

      Tried to access the job configuration from UI, when the job history files were still in the intermediate directory. JHS displayed the configurations of the job. Again tried to access the configurations of the same job, when the job history files were in the done directory. This time got the following exception

      java.io.FileNotFoundException: File does not exist: /jobhistory/intermediate/kamesh/job_1332999698561_0005_conf.xml
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1153)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1115)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1087)
      

        Issue Links

          Activity

          Hide
          Robert Joseph Evans added a comment -

          I saw this same issue while working on MAPREDUCE-3972, and I fixed it as part of that JIRA. I did this because I considered it a threading issue. The move from done_intermediate to done was not coordinated with the reading of the Configuration file. It also was not coordinated with reading the jhist file or many other things.

          Show
          Robert Joseph Evans added a comment - I saw this same issue while working on MAPREDUCE-3972 , and I fixed it as part of that JIRA. I did this because I considered it a threading issue. The move from done_intermediate to done was not coordinated with the reading of the Configuration file. It also was not coordinated with reading the jhist file or many other things.
          Hide
          Bhallamudi Venkata Siva Kamesh added a comment -

          This is because of the stale object returned from the loadedJobCache.

          When I tried to access the job config first time, when job history files were present in the intermediate directory, as loadedJobCache does not contain any entries to fulfill the request, CompletedJob object was created and has been added to loadedJobCache and the same object has been returned. However when I tried to access the same job config again, this time job history files were present in the done directory, as loadedJobCache already contained the job entry, loadedJobCache returned the job object from its cache. So the returned job object's confFile path pointing to intermediate directory where as the job's actual confFile is present inside done directory and hence the exception.

          Show
          Bhallamudi Venkata Siva Kamesh added a comment - This is because of the stale object returned from the loadedJobCache . When I tried to access the job config first time, when job history files were present in the intermediate directory, as loadedJobCache does not contain any entries to fulfill the request, CompletedJob object was created and has been added to loadedJobCache and the same object has been returned. However when I tried to access the same job config again, this time job history files were present in the done directory, as loadedJobCache already contained the job entry, loadedJobCache returned the job object from its cache. So the returned job object's confFile path pointing to intermediate directory where as the job's actual confFile is present inside done directory and hence the exception.

            People

            • Assignee:
              Unassigned
              Reporter:
              Bhallamudi Venkata Siva Kamesh
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development