Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9991

Fix up Hadoop POMs, roll up JARs to latest versions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.1-beta, 2.3.0
    • None
    • build
    • None

    Description

      If you try using Hadoop downstream with a classpath shared with HBase and Accumulo, you soon discover how messy the dependencies are.

      Hadoop's side of this problem is

      1. not being up to date with some of the external releases of common JARs
      2. not locking down/excluding inconsistent versions of artifacts provided down the dependency graph

      Attachments

        1. hadoop-9991-v1.txt
          5 kB
          Ted Yu

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HADOOP-13050 Upgrade to AWS SDK 1.11.45

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14890 Move up to AWS SDK 1.11.199

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15642 Update aws-sdk version to 1.11.375

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-9613 [JDK8] Update jersey version to latest 1.x release

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14040 Use shaded aws-sdk uber-JAR 1.11.86

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14490 Upgrade azure-storage sdk version >5.4.0

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14775 Change junit dependency in parent pom file to junit 5 while maintaining backward compatibility to junit4.

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-13541 explicitly declare the Joda time version S3A depends on

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14126 remove jackson, joda and other transient aws SDK dependencies from hadoop-aws

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-5411 Update Bookkeeper dependency to 4.2.3

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14100 Upgrade Jsch jar to latest version to fix vulnerability in old versions

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14283 Upgrade AWS SDK to 1.11.134

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15960 Update guava to 27.0-jre in hadoop-project

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11515 Upgrade jsch lib to jsch-0.1.51 to avoid problems running on java7

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-12767 update apache httpclient version to 4.5.2; httpcore to 4.4.4

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-13994 explicitly declare the commons-lang3 dependency as 3.4

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14824 Update ADLS SDK to 2.2.2 for MSI fix

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15054 upgrade hadoop dependency on commons-codec to 1.11

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15261 Upgrade commons-io from 2.4 to 2.5

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15299 Bump Hadoop's Jackson 2 dependency 2.9.x

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-16076 SPNEGO+SSL Client Connections with HttpClient Broken

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11086 Upgrade jets3t to 0.9.4

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11946 Update jets3t jar to 0.9.4 in order to fix leading '/' problem with S3 calls.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-13545 Upgrade HSQLDB to 2.3.4

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14420 generateReports property is not applicable for maven-site-plugin:attach-descriptor goal

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14692 Upgrade Apache Rat

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-12577 Bump up commons-collections version to 3.2.2 to address a security flaw

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9555 HA functionality that uses ZooKeeper may experience inadvertent TCP RST and miss session expiration event due to bug in client connection management

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10814 Update Tomcat version used by HttpFS and KMS to latest 6.x version

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5431 Missing pom dependency in MR-client

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10147 Upgrade to commons-logging 1.1.3 to avoid potential deadlock in MiniDFSCluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-12281 Add an incompatible version of Kryo to the hadoop-client classpath

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10076 Update tomcat/jasper dependency to version 7

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13363 Upgrade protobuf from 2.5.0 to something newer

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12064 [JDK8] Update guice version to 4.0

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10075 Update jetty dependency to version 9

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10101 Update guava dependency to the latest version

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12527 Upgrade Avro dependency to 1.7.7 or later

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12927 Update netty-all to 4.0.34.Final

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12928 Update netty to 3.10.5.Final to sync with zookeeper

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13866 Upgrade netty-all to 4.1.1.Final

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14401 maven-project-info-reports-plugin can be removed

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14471 Upgrade Jetty to latest 9.3 version

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14536 Update azure-storage sdk to version 5.3.0

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14662 Update azure-storage sdk to version 5.4.0

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14930 Upgrade Jetty to 9.4 version

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15072 Upgrade Apache Kerby version to 2.0.x

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-16152 Upgrade Eclipse Jetty version to 9.4.x

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5624 move grizzly-test and junit dependencies to test scope

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5678 Move junit to test scope in more projects

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9244 Upgrade servlet-api dependency from version 2.5 to 3.0.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10384 CLONE - Upgrade servlet-api dependency from version 2.5 to 3.0.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12427 [JDK8] Upgrade Mockito version to 1.10.19

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14414 Calling maven-site-plugin directly for docs profile is unnecessary

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14419 Remove findbugs report from docs profile

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-16075 Upgrade checkstyle version to 8.16

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9594 Update apache commons math dependency

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9611 mvn-rpmbuild against google-guice > 3.0 yields missing cglib dependency

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9833 move slf4j to version 1.7.5

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. YARN-3774 ZKRMStateStore should use CuratorOp when we upgrade to Curator 3

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Patch Available
          incorporates

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13659 Upgrade jaxb-api version

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13660 Upgrade commons-configuration version to 2.1

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13661 Upgrade HTrace version

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HADOOP-15482 Upgrade jackson-databind to version 2.9.5

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11755 Update avro version to have PowerPC supported Snappy-java

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is related to

          Sub-task - The sub-task of the issue HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE in 1.11.199 bug spams logs w/ Yarn Log Aggregation

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14290 Update SLF4J from 1.7.10 to 1.7.25

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15214 Make Hadoop compatible with Guava 21.0

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. BOOKKEEPER-708 Shade protobuf library to avoid incompatible versions

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9905 remove dependency of zookeeper for hadoop-client

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10783 apache-commons-lang.jar 2.6 does not support FreeBSD -upgrade to 3.x needed

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-8793 hadoop-core has dependencies on two different versions of commons-httpclient

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10100 MiniKDC shouldn't use apacheds-all artifact

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9961 versions of a few transitive dependencies diverged between hadoop subprojects

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10105 remove httpclient dependency

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14043 Shade netty 4 dependency in hadoop-hdfs

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14061 Update checkstyle version to 6.16 or upper

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14670 Increase minimum cmake version for all platforms

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15993 Upgrade Kafka version in hadoop-kafka module

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15994 Upgrade Jackson2 to 2.9.8

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-16227 Upgrade checkstyle to 8.19

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12809 Move to tomcat 8.0.21 to support different passwords for key and keystore on HTTPFS/KMS using SSL

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10530 Make hadoop trunk build on Java7+ only

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12232 Upgrade Tomcat dependency to 6.0.44.

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10067 Missing POM dependency on jsr305

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Task - A task that needs to be done. HADOOP-14647 Update third-party libraries for Hadoop 3

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-15960 Update guava to 27.0-jre in hadoop-project

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15218 Make Hadoop compatible with Guava 22.0+

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HADOOP-11492 Bump up curator version to 2.7.1

          • Major - Major loss of function.
          • Closed
          requires

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13332 Remove jackson 1.9.13 and switch all jackson code to 2.x code line

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #85

          1.
          		 update commons IO from 2.1 to 2.4 Sub-task Closed Akira Ajisaka  
          2.
          		 update commons-lang to 2.6 Sub-task Closed Akira Ajisaka  
          3.
          		 Update jackson to 1.9.13 Sub-task Closed Akira Ajisaka  
          4.
          		 Move junit up to v 4.11 Sub-task Closed Chris Nauroth  
          5.
          		 Bump the maven plugin versions too -moving the numbers into properties Sub-task Closed Akira Ajisaka  
          6.
          		 Increment SLF4J version to 1.7.10 Sub-task Closed Tim Robertson  
          7.
          		 Update Guava to 18.0 Sub-task Resolved Unassigned  
          8.
          		 Update aws-sdk dependency to 1.10.6; move to aws-sdk-s3 Sub-task Resolved Thomas Demoor  
          9.
          		 Upgrade Jackson 2.2.3 to 2.7.8 Sub-task Resolved Sean Mackrory  
          10.
          		 [JDK8] Update jersey version to latest 1.x release Sub-task Resolved Tsuyoshi Ozawa  
          11.
          		 Update jetty dependencies Sub-task Resolved Timothy St. Clair  
          12.
          		 clean up POM dependencies Sub-task Open Alejandro Abdelnur  
          13.
          		 Move to netty 4.1.x release Sub-task Resolved Unassigned  
          14.
          		 shade protobuf in the hadoop-common jar Sub-task Resolved Unassigned  
          15.
          		 Upgrade Avro to 1.9.2 Sub-task Resolved PJ Fanning

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 7h
          16.
          		 Move Mockito up to version 2.23.4 Sub-task Resolved Akira Ajisaka  
          17.
          		 Update commons-net to 3.6 Sub-task Open Kevin Su  
          18.
          		 Update metrics-core version to 3.2.4 Sub-task Resolved Ray Chiang  
          19.
          		 Update okhttp version to 2.7.5 Sub-task Resolved Ray Chiang  
          20.
          		 Move Mockito up to 1.10.19 to be compatible with HBase Sub-task Resolved Unassigned  
          21.
          		 Move commons-net up to 3.6 Sub-task Resolved Steve Loughran  
          22.
          		 Update AssertJ to 3.12.2 Sub-task Resolved Takanobu Asanuma  
          23.
          		 Update okhttp to 3.14.1 Sub-task Resolved Lisheng Sun  
          24.
          		 Update HikariCP to 4.0.3 Sub-task Resolved Viraj Jasani

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          25.
          		 Update mssql-jdbc to 7.2.2.jre8 Sub-task Patch Available Lisheng Sun  
          26.
          		 Upgrade Mockito to the latest version Sub-task Resolved Kevin Su  
          27.
          		 Bump Apache Avro to 1.9.0 Sub-task Resolved Unassigned  
          28.
          		 Upgrade Wildfly Openssl version to 1.0.7.Final Sub-task Resolved Vishwajeet Dusane  
          29.
          		 Update to Apache LDAP API 2.0.x Sub-task Patch Available Colm O hEigeartaigh

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              6 Vote for this issue
              Watchers:
              44 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 8h 40m
                  8h 40m