Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13204 Über-jira: S3a phase III: scale and tuning
  3. HADOOP-14040

Use shaded aws-sdk uber-JAR 1.11.86

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0
    • 2.9.0, 3.0.0-alpha4
    • build, fs/s3
    • None
    • Reviewed

    Description

      AWS SDK now has a (v. large) uberjar shading all dependencies

      This ensures that AWS dependency changes (e.g json) don't cause problems downstream in things like HBase, so enabling backporting if desired.

      This will let us addess the org.json don't be evil problem: this SDK version doesn't have those files.

      Attachments

        1. HADOOP-14040-branch-2.002.patch
          2 kB
          Sean Mackrory
        2. HADOOP-14040-HADOOP-13345.001.patch
          3 kB
          Sean Mackrory
        3. HADOOP-14040.001.patch
          2 kB
          Sean Mackrory
        4. HADOOP-14040-branch-2-001.patch
          2 kB
          Steve Loughran

        Issue Links

          breaks

          Sub-task - The sub-task of the issue HADOOP-14138 Remove S3A ref from META-INF service discovery, rely on existing core-default entry

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. AMBARI-21665 shaded AWS SDK in Hadoop JARs slowing ambari view loading

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9991 Fix up Hadoop POMs, roll up JARs to latest versions

          • Major - Major loss of function.
          • Open
          is related to

          Sub-task - The sub-task of the issue HADOOP-14890 Move up to AWS SDK 1.11.199

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-17337 S3A NetworkBinding has a runtime class dependency on a third-party shaded class

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14126 remove jackson, joda and other transient aws SDK dependencies from hadoop-aws

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-15499 Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: