Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14269 Performance optimizations for data on S3
  3. HIVE-16276

Fix NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Druid integration
    • None

    Description

      The druid-handler is pulling in some com.amazonaws dependencies that conflict with the version that Hadoop is using. This causes the above exception to be thrown when running Hive against S3. This patch fixes the dependency issue by shading the aws dependencies in the druid artifacts. Unfortunately, I can't find a great way to add a test for this so it doesn't happen in the future. We will need some more robust S3-integration tests for that.

      Attachments

        1. HIVE-16276.1.patch
          3 kB
          Sahil Takiar
        2. HIVE-16276.2.patch
          3 kB
          Sahil Takiar
        3. HIVE-16276.3.patch
          2 kB
          Sahil Takiar
        4. HIVE-16276.4.patch
          3 kB
          Sahil Takiar
        5. HIVE-16276.5.patch
          0.5 kB
          Sahil Takiar

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jcamachorodriguez Jesus Camacho Rodriguez Assign to me
            stakiar Sahil Takiar
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment