Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4864

-libjars with multiple jars broken when client and cluster reside on different OSs

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.21.0
    • Component/s: filecache
    • Labels:
      None
    • Environment:

      When your hadoop job spans OSs.

    • Hadoop Flags:
      Reviewed

      Description

      When submitting a hadoop job from Windows (Cygwin) to a Linux hadoop cluster (or vice versa), and when you specify multiple additional jar files via the -libjars flag, hadoop throws a ClassNotFoundException for any classes located in the additional jars specified via the -libjars flag.

      This is caused by the fact that hadoop uses system.getProperty("path.separator") as the delimiter in the list of jar files passed via -libjars.

      My suggested solution is to use a comma as the delimiter, rather than the path.separator.

      I realize comma is, perhaps, a poor choice for a delimiter because it is valid in filenames on both Windows and Linux, but the -libjars flag uses it as the delimiter when listing the additional required jars. So, I figured if it's already being used as a delimiter, then it's reasonable to use it internally as well.

        Attachments

        1. patch-4864-1.txt
          3 kB
          Amareshwari Sriramadasu
        2. patch-4864.txt
          2 kB
          Amareshwari Sriramadasu
        3. patch.txt
          1 kB
          Stuart White

          Activity

            People

            • Assignee:
              amareshwari Amareshwari Sriramadasu
              Reporter:
              stwhit Stuart White
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified