Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3428

MR AppMaster CLASSPATH is dependent on the compile-time environment

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:

      Description

      The CLASSPATH for the MapReduce Application master is set using compile time path information, which is typically different from run-time. This will cause failure when running on different environments.

      Specifically, the YarnRunner, and as part ApplicationSubmissionContext creation, sets the classpath for the application master using MRApps.setClasspath(environment), and then the setMRFrameworkClasspath(..) method uses compile time path information present in the "mrapp-generated-classpath" file (created at compile-time).

        Issue Links

          Activity

          Ahmed Radwan created issue -
          Ahmed Radwan made changes -
          Field Original Value New Value
          Description The CLASSPATH for the MapReduce Application master is set using compile time path information, which is typically different from run-time. This will cause failure when running on different environments.

          Specifically, the YarnRunner, and as part ApplicationSubmissionContext creation, sets the classpath for the application master using MRApps.setClasspath(environment), and then the setMRFrameworkClasspath(..) method uses compile time path information present in the "setMRFrameworkClasspath" file (created at compile-time).
          The CLASSPATH for the MapReduce Application master is set using compile time path information, which is typically different from run-time. This will cause failure when running on different environments.

          Specifically, the YarnRunner, and as part ApplicationSubmissionContext creation, sets the classpath for the application master using MRApps.setClasspath(environment), and then the setMRFrameworkClasspath(..) method uses compile time path information present in the "mrapp-generated-classpath" file (created at compile-time).
          Hide
          Alejandro Abdelnur added a comment -

          This seems a dup of MAPREDUCE-3428 (if so please close one of them)

          Show
          Alejandro Abdelnur added a comment - This seems a dup of MAPREDUCE-3428 (if so please close one of them)
          Hide
          Ravi Prakash added a comment -

          Tucu probably means MAPREDUCE-3389. Right?

          Show
          Ravi Prakash added a comment - Tucu probably means MAPREDUCE-3389 . Right?
          Hide
          Alejandro Abdelnur added a comment -

          Yes (C&P typo), thanks Ravi.

          Show
          Alejandro Abdelnur added a comment - Yes (C&P typo), thanks Ravi.
          Tom White made changes -
          Link This issue duplicates MAPREDUCE-3389 [ MAPREDUCE-3389 ]
          Tom White made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Alejandro Abdelnur made changes -
          Link This issue is related to MAPREDUCE-3509 [ MAPREDUCE-3509 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is duplicated by MAPREDUCE-3389 [ MAPREDUCE-3389 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue duplicates MAPREDUCE-3389 [ MAPREDUCE-3389 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          14d 9h 46m 1 Tom White 02/Dec/11 19:51

            People

            • Assignee:
              Unassigned
              Reporter:
              Ahmed Radwan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development