Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-129

Simplify classpath construction for mini YARN tests

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha
    • Component/s: client
    • Labels:
      None

      Description

      The test classpath includes a special file called 'mrapp-generated-classpath' (or similar in distributed shell) that is constructed at build time, and whose contents are a classpath with all the dependencies needed to run the tests. When the classpath for a container (e.g. the AM) is constructed the contents of mrapp-generated-classpath is read and added to the classpath, and the file itself is then added to the classpath so that later when the AM constructs a classpath for a task container it can propagate the test classpath correctly.

      This mechanism can be drastically simplified by propagating the system classpath of the current JVM (read from the java.class.path property) to a launched JVM, but only if running in the context of the mini YARN cluster. Any tests that use the mini YARN cluster will automatically work with this change. Although any that explicitly deal with mrapp-generated-classpath can be simplified.

        Attachments

        1. YARN-129.patch
          18 kB
          Thomas White
        2. YARN-129.patch
          18 kB
          Thomas White
        3. YARN-129.patch
          17 kB
          Thomas White

          Issue Links

            Activity

              People

              • Assignee:
                tomwhite Thomas White
                Reporter:
                tomwhite Thomas White
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: