Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10028 Additional optimizations of Impala docker container sizes
  3. IMPALA-10073

Create shaded dependency for S3A and aws-java-sdk-bundle

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.0.0
    • None
    • None
    • ghx-label-5

    Description

      One of the largest dependencies in Impala Docker containers is the aws-java-sdk-bundle jar. One way to decrease the size of this dependency is to apply a similar technique used for the hive-exec shaded jar: https://github.com/apache/impala/blob/master/shaded-deps/pom.xml

      The aws-java-sdk-bundle contains SDKs for all AWS services, even though Impala-S3A only requires a few of the more basic SDKs.

      IMPALA-10028 and HADOOP-17197 both discuss this a bit as well.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HADOOP-17337 S3A NetworkBinding has a runtime class dependency on a third-party shaded class

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Activity

            People

              stakiar Sahil Takiar
              stakiar Sahil Takiar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: