Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8899

Classpath exceeds maximum OS limit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1-win
    • Fix Version/s: 1-win
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We hit errors in TT due to long classpath value. One example is Oozie trying to start a hive job, and it fails with the following error:
      java.io.IOException: Command exceeds the OS command length limit: 8192, command: "set CLASSPATH="….

      The classpath includes Hadoop Core + Hive Jars which are in the distributed cache. This is causing the classpath to be too long giving the error above. A viable long term fix is to generate a temporary JAR file in the task cache directory which includes all elements in the long classpath, and then provide this as the "-classpath" argument for the JVM to be spawned. This is gated for Windows only.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ahelbaz Ahmed El Baz
                Reporter:
                ahelbaz Ahmed El Baz
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: