Hadoop Common
  1. Hadoop Common
  2. HADOOP-8327

distcpv2 and distcpv1 jars should not coexist

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Resolve sporadic distcp issue due to having two DistCp classes (v1 & v2) in the classpath.
    • Tags:
      distcp distcpv1 distcpv2 classpath

      Description

      Distcp v2 (hadoop-tools/hadoop-distcp/...) and Distcp v1 (hadoop-tools/hadoop-extras/...) are currently both built, and the resulting hadoop-distcp-x.jar and hadoop-extras-x.jar end up in the same class path directory. This causes some undeterministic problems, where v1 is launched when v2 is intended, or even v2 is launched, but may later fail on various nodes because of mismatch with v1.

      According to
      http://docs.oracle.com/javase/6/docs/technotes/tools/windows/classpath.html ("Understanding class path wildcards")

      "The order in which the JAR files in a directory are enumerated in the expanded class path is not specified and may vary from platform to platform and even from moment to moment on the same machine."

      Suggest distcpv1 be deprecated at this point, possibly by discontinuing build of distcpv1.

      1. HADOOP-8327-branch-0.23.2.patch
        142 kB
        Dave Thompson
      2. HADOOP-8327.patch
        142 kB
        Dave Thompson
      3. HADOOP-8327.patch
        142 kB
        Dave Thompson

        Activity

          People

          • Assignee:
            Dave Thompson
            Reporter:
            Dave Thompson
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development