Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10115

Exclude duplicate jars in hadoop package under different component's lib

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Incompatible change
    • Jars in the various subproject lib directories are now de-duplicated against Hadoop common. Users who interact directly with those directories must be sure to pull in common's dependencies as well.

    Description

      In the hadoop package distribution there are more than 90% of the jars are duplicated in multiple places.
      For Ex:
      almost all jars in share/hadoop/hdfs/lib are already there in share/hadoop/common/lib

      Same case for all other lib in share directory.

      Anyway for all the daemon processes all directories are added to classpath.

      So to reduce the package distribution size and the classpath overhead, remove the duplicate jars from the distribution.

      Attachments

        1. HADOOP-10115.patch
          5 kB
          Vinayakumar B
        2. HADOOP-10115.patch
          4 kB
          Vinayakumar B
        3. HADOOP-10115.patch
          5 kB
          Vinayakumar B
        4. HADOOP-10115-004.patch
          5 kB
          Vinayakumar B
        5. HADOOP-10115-005.patch
          6 kB
          Vinayakumar B
        6. HADOOP-10115-006.patch
          6 kB
          Vinayakumar B
        7. HADOOP-10115-007.patch
          6 kB
          Vinayakumar B

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vinayakumarb Vinayakumar B
            vinayakumarb Vinayakumar B
            Votes:
            0 Vote for this issue
            Watchers:
            12 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment