Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8899

Classpath exceeds maximum OS limit

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1-win
    • 1-win
    • None
    • None
    • Reviewed

    Description

      We hit errors in TT due to long classpath value. One example is Oozie trying to start a hive job, and it fails with the following error:
      java.io.IOException: Command exceeds the OS command length limit: 8192, command: "set CLASSPATH="….

      The classpath includes Hadoop Core + Hive Jars which are in the distributed cache. This is causing the classpath to be too long giving the error above. A viable long term fix is to generate a temporary JAR file in the task cache directory which includes all elements in the long classpath, and then provide this as the "-classpath" argument for the JVM to be spawned. This is gated for Windows only.

      Attachments

        Issue Links

          Activity

            People

              ahelbaz Ahmed El Baz
              ahelbaz Ahmed El Baz
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: