Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8327

distcpv2 and distcpv1 jars should not coexist

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Resolve sporadic distcp issue due to having two DistCp classes (v1 & v2) in the classpath.
    • Tags:
      distcp distcpv1 distcpv2 classpath

      Description

      Distcp v2 (hadoop-tools/hadoop-distcp/...) and Distcp v1 (hadoop-tools/hadoop-extras/...) are currently both built, and the resulting hadoop-distcp-x.jar and hadoop-extras-x.jar end up in the same class path directory. This causes some undeterministic problems, where v1 is launched when v2 is intended, or even v2 is launched, but may later fail on various nodes because of mismatch with v1.

      According to
      http://docs.oracle.com/javase/6/docs/technotes/tools/windows/classpath.html ("Understanding class path wildcards")

      "The order in which the JAR files in a directory are enumerated in the expanded class path is not specified and may vary from platform to platform and even from moment to moment on the same machine."

      Suggest distcpv1 be deprecated at this point, possibly by discontinuing build of distcpv1.

        Attachments

        1. HADOOP-8327-branch-0.23.2.patch
          142 kB
          Dave Thompson
        2. HADOOP-8327.patch
          142 kB
          Dave Thompson
        3. HADOOP-8327.patch
          142 kB
          Dave Thompson

          Activity

            People

            • Assignee:
              davet Dave Thompson
              Reporter:
              davet Dave Thompson
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: