Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: build, fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      The dependency on the AWS SDK has been bumped to 1.11.45.

      Description

      HADOOP-13044 highlights that AWS SDK 10.6 —shipping in Hadoop 2.7+, doesn't work on open jdk >= 8u60, because a change in the JDK broke the version of Joda time that AWS uses.

      Fix, update the JDK. Though, that implies updating http components: HADOOP-12767.

      1. HADOOP-13050-branch-2-004.patch
        5 kB
        Steve Loughran
      2. HADOOP-13050-002.patch
        6 kB
        Mingliang Liu
      3. HADOOP-13050-branch-2-003.patch
        6 kB
        Steve Loughran
      4. HADOOP-13050-branch-2.003.patch
        6 kB
        Chris Nauroth
      5. HADOOP-13050-branch-2.002.patch
        2 kB
        Chris Nauroth
      6. HADOOP-13050-001.patch
        2 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'll make this patch something where you can set the aws version on the command line; that way people can build hadoop with their own choice of version without having to edit the poms.

          Dependencies of the 1.10.77 full SDK below

          • there's a new jackson dependency, jackson-dataformat-cbor not used anywhere in hadoop —so not under hadoop's version management. It' needs to be explicitly declared in hadoop-project and then depended on in hadoop-aws.
          • yes, a new joda-time
          • the httpcomponents are as 10.6 as before. Meaning HADOOP-12767 exists in Hadoop 2.8 today
          [INFO] +- com.amazonaws:aws-java-sdk:jar:1.10.77:compile
          [INFO] |  +- com.amazonaws:aws-java-sdk-support:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-simpledb:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-storagegateway:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-route53:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-importexport:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-sts:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-rds:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-redshift:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-elasticbeanstalk:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-glacier:jar:1.10.77:compile
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-iam:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-datapipeline:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-elasticloadbalancing:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-emr:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-elasticache:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-elastictranscoder:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-ec2:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudtrail:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-logs:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-events:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cognitoidentity:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cognitosync:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-directconnect:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudformation:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudfront:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-kinesis:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-opsworks:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-ses:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-autoscaling:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudsearch:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudwatchmetrics:jar:1.10.77:compile
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  +- (com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-swf-libraries:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-codedeploy:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-codepipeline:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-config:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-lambda:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-ecs:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-ecr:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cloudhsm:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-ssm:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-workspaces:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-machinelearning:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-directory:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-efs:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-codecommit:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-devicefarm:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-elasticsearch:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-waf:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-marketplacecommerceanalytics:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-inspector:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-iot:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-api-gateway:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-acm:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-gamelift:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-dms:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-marketplacemeteringservice:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  +- com.amazonaws:aws-java-sdk-cognitoidp:jar:1.10.77:compile
          [INFO] |  |  \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate)
          [INFO] |  \- com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile
          [INFO] |     +- (commons-logging:commons-logging:jar:1.1.3:compile - version managed from 1.1.1; omitted for duplicate)
          [INFO] |     +- (org.apache.httpcomponents:httpclient:jar:4.2.5:compile - version managed from 4.3.6; omitted for duplicate)
          [INFO] |     +- (com.fasterxml.jackson.core:jackson-databind:jar:2.2.3:compile - version managed from 2.5.3; omitted for duplicate)
          [INFO] |     +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile
          [INFO] |     |  \- (com.fasterxml.jackson.core:jackson-core:jar:2.2.3:compile - version managed from 2.5.3; omitted for duplicate)
          [INFO] |     \- joda-time:joda-time:jar:2.8.1:compile
          
          Show
          stevel@apache.org Steve Loughran added a comment - I'll make this patch something where you can set the aws version on the command line; that way people can build hadoop with their own choice of version without having to edit the poms. Dependencies of the 1.10.77 full SDK below there's a new jackson dependency, jackson-dataformat-cbor not used anywhere in hadoop —so not under hadoop's version management. It' needs to be explicitly declared in hadoop-project and then depended on in hadoop-aws. yes, a new joda-time the httpcomponents are as 10.6 as before. Meaning HADOOP-12767 exists in Hadoop 2.8 today [INFO] +- com.amazonaws:aws-java-sdk:jar:1.10.77:compile [INFO] | +- com.amazonaws:aws-java-sdk-support:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-simpledb:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-storagegateway:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-route53:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile [INFO] | | +- (com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile - omitted for duplicate) [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-importexport:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-sts:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-rds:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-redshift:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-elasticbeanstalk:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-glacier:jar:1.10.77:compile [INFO] | | +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted for duplicate) [INFO] | | +- (com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile - omitted for duplicate) [INFO] | | +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted for duplicate) [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-iam:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-datapipeline:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-elasticloadbalancing:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-emr:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-elasticache:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-elastictranscoder:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-ec2:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile [INFO] | | +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted for duplicate) [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile [INFO] | | +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted for duplicate) [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudtrail:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-logs:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-events:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cognitoidentity:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cognitosync:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-directconnect:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudformation:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudfront:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-kinesis:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-opsworks:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-ses:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-autoscaling:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudsearch:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudwatchmetrics:jar:1.10.77:compile [INFO] | | +- (com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile - omitted for duplicate) [INFO] | | +- (com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile - omitted for duplicate) [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-swf-libraries:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-codedeploy:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-codepipeline:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-config:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-lambda:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-ecs:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-ecr:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cloudhsm:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-ssm:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-workspaces:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-machinelearning:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-directory:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-efs:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-codecommit:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-devicefarm:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-elasticsearch:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-waf:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-marketplacecommerceanalytics:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-inspector:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-iot:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-api-gateway:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-acm:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-gamelift:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-dms:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-marketplacemeteringservice:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | +- com.amazonaws:aws-java-sdk-cognitoidp:jar:1.10.77:compile [INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted for duplicate) [INFO] | \- com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile [INFO] | +- (commons-logging:commons-logging:jar:1.1.3:compile - version managed from 1.1.1; omitted for duplicate) [INFO] | +- (org.apache.httpcomponents:httpclient:jar:4.2.5:compile - version managed from 4.3.6; omitted for duplicate) [INFO] | +- (com.fasterxml.jackson.core:jackson-databind:jar:2.2.3:compile - version managed from 2.5.3; omitted for duplicate) [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile [INFO] | | \- (com.fasterxml.jackson.core:jackson-core:jar:2.2.3:compile - version managed from 2.5.3; omitted for duplicate) [INFO] | \- joda-time:joda-time:jar:2.8.1:compile
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 001 Move AWS SDK up to 1.10.77 (last of the 1.10 releases). This includes HADOOP-12705 -increasing Jackson to 2.5.5

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 001 Move AWS SDK up to 1.10.77 (last of the 1.10 releases). This includes HADOOP-12705 -increasing Jackson to 2.5.5
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This patch increments the AWS library version, and explicitly manages the jackson.databind dependency, which forces us to increment the entire jackson dependency to 2.3.3 or later. I've gone with a late 2.5.x as they're in use elsewhere.

          Tagging as incompatible, as we may warn people that all JAR updates are trouble

          Show
          stevel@apache.org Steve Loughran added a comment - This patch increments the AWS library version, and explicitly manages the jackson.databind dependency, which forces us to increment the entire jackson dependency to 2.3.3 or later. I've gone with a late 2.5.x as they're in use elsewhere. Tagging as incompatible, as we may warn people that all JAR updates are trouble
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 8m 18s trunk passed
          +1 compile 8m 9s trunk passed
          +1 mvnsite 0m 26s trunk passed
          +1 mvneclipse 0m 23s trunk passed
          +1 javadoc 0m 21s trunk passed
          0 mvndep 0m 20s Maven dependency ordering for patch
          +1 mvninstall 0m 19s the patch passed
          +1 compile 7m 37s the patch passed
          -1 javac 7m 37s root generated 2 new + 698 unchanged - 0 fixed = 700 total (was 698)
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 24s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 javadoc 0m 20s the patch passed
          +1 unit 0m 8s hadoop-project in the patch passed.
          +1 unit 0m 15s hadoop-aws in the patch passed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          28m 54s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804955/HADOOP-13050-001.patch
          JIRA Issue HADOOP-13050
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 8d898e954695 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 141873c
          Default Java 1.8.0_91
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/artifact/patchprocess/diff-compile-javac-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 8m 18s trunk passed +1 compile 8m 9s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 23s trunk passed +1 javadoc 0m 21s trunk passed 0 mvndep 0m 20s Maven dependency ordering for patch +1 mvninstall 0m 19s the patch passed +1 compile 7m 37s the patch passed -1 javac 7m 37s root generated 2 new + 698 unchanged - 0 fixed = 700 total (was 698) +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 javadoc 0m 20s the patch passed +1 unit 0m 8s hadoop-project in the patch passed. +1 unit 0m 15s hadoop-aws in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 28m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804955/HADOOP-13050-001.patch JIRA Issue HADOOP-13050 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 8d898e954695 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 141873c Default Java 1.8.0_91 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/artifact/patchprocess/diff-compile-javac-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9515/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Note that I'm not seeing this fail on the version of java 8u9x I've got installed locally. It appears that what matters is having a Joda time > 2.8.0 which makes the difference

          Show
          stevel@apache.org Steve Loughran added a comment - Note that I'm not seeing this fail on the version of java 8u9x I've got installed locally. It appears that what matters is having a Joda time > 2.8.0 which makes the difference
          Hide
          cnauroth Chris Nauroth added a comment -

          Linking to HADOOP-13727, which is a workaround within hadoop-aws that we might be able to remove after upgrading the AWS SDK.

          Show
          cnauroth Chris Nauroth added a comment - Linking to HADOOP-13727 , which is a workaround within hadoop-aws that we might be able to remove after upgrading the AWS SDK.
          Hide
          cnauroth Chris Nauroth added a comment -

          This needed a rebase after one of my patches, so I went ahead and updated it. I'm attaching revision 002. This time, I went to AWS SDK 1.11.45. All tests passed against US-west-2.

          Here is the updated dependency tree:

          [INFO] --- maven-dependency-plugin:2.2:tree (default-cli) @ hadoop-aws ---
          [INFO] org.apache.hadoop:hadoop-aws:jar:2.9.0-SNAPSHOT
          [INFO] +- org.apache.hadoop:hadoop-common:jar:2.9.0-SNAPSHOT:compile
          [INFO] |  +- org.apache.hadoop:hadoop-annotations:jar:2.9.0-SNAPSHOT:compile
          [INFO] |  |  \- jdk.tools:jdk.tools:jar:1.7:system
          [INFO] |  +- com.google.guava:guava:jar:11.0.2:compile
          [INFO] |  +- commons-cli:commons-cli:jar:1.2:compile
          [INFO] |  +- org.apache.commons:commons-math3:jar:3.1.1:compile
          [INFO] |  +- xmlenc:xmlenc:jar:0.52:compile
          [INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
          [INFO] |  |  \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
          [INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
          [INFO] |  +- commons-io:commons-io:jar:2.4:compile
          [INFO] |  +- commons-net:commons-net:jar:3.1:compile
          [INFO] |  +- commons-collections:commons-collections:jar:3.2.2:compile
          [INFO] |  +- javax.servlet:servlet-api:jar:2.5:compile
          [INFO] |  +- org.mortbay.jetty:jetty:jar:6.1.26:compile
          [INFO] |  +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
          [INFO] |  +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26:compile
          [INFO] |  +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
          [INFO] |  +- com.sun.jersey:jersey-core:jar:1.9:compile
          [INFO] |  +- com.sun.jersey:jersey-json:jar:1.9:compile
          [INFO] |  |  +- org.codehaus.jettison:jettison:jar:1.1:compile
          [INFO] |  |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
          [INFO] |  |  |  \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
          [INFO] |  |  |     +- javax.xml.stream:stax-api:jar:1.0-2:compile
          [INFO] |  |  |     \- javax.activation:activation:jar:1.1:compile
          [INFO] |  |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile (version managed from 1.8.3)
          [INFO] |  |  \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile (version managed from 1.8.3)
          [INFO] |  +- com.sun.jersey:jersey-server:jar:1.9:compile
          [INFO] |  |  \- asm:asm:jar:3.2:compile (version managed from 3.1)
          [INFO] |  +- commons-logging:commons-logging:jar:1.1.3:compile
          [INFO] |  +- log4j:log4j:jar:1.2.17:compile
          [INFO] |  +- net.java.dev.jets3t:jets3t:jar:0.9.0:compile
          [INFO] |  |  \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile
          [INFO] |  +- commons-lang:commons-lang:jar:2.6:compile
          [INFO] |  +- commons-configuration:commons-configuration:jar:1.6:compile
          [INFO] |  |  +- commons-digester:commons-digester:jar:1.8:compile
          [INFO] |  |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
          [INFO] |  |  \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
          [INFO] |  +- org.slf4j:slf4j-api:jar:1.7.10:compile
          [INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
          [INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
          [INFO] |  +- org.apache.avro:avro:jar:1.7.4:compile
          [INFO] |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
          [INFO] |  |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
          [INFO] |  +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
          [INFO] |  +- com.google.code.gson:gson:jar:2.2.4:compile
          [INFO] |  +- org.apache.hadoop:hadoop-auth:jar:2.9.0-SNAPSHOT:compile
          [INFO] |  |  +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile
          [INFO] |  |  |  +- net.jcip:jcip-annotations:jar:1.0:compile
          [INFO] |  |  |  \- net.minidev:json-smart:jar:1.1.1:compile
          [INFO] |  |  +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
          [INFO] |  |  |  +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
          [INFO] |  |  |  +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
          [INFO] |  |  |  \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
          [INFO] |  |  \- org.apache.curator:curator-framework:jar:2.7.1:compile
          [INFO] |  +- com.jcraft:jsch:jar:0.1.51:compile
          [INFO] |  +- org.apache.curator:curator-client:jar:2.7.1:compile
          [INFO] |  +- org.apache.curator:curator-recipes:jar:2.7.1:compile
          [INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.0:compile
          [INFO] |  +- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile
          [INFO] |  +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
          [INFO] |  |  +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile (version managed from 1.6.1)
          [INFO] |  |  \- io.netty:netty:jar:3.6.2.Final:compile (version managed from 3.7.0.Final)
          [INFO] |  \- org.apache.commons:commons-compress:jar:1.4.1:compile
          [INFO] |     \- org.tukaani:xz:jar:1.0:compile
          [INFO] +- org.apache.hadoop:hadoop-common:test-jar:tests:2.9.0-SNAPSHOT:test
          [INFO] +- com.amazonaws:aws-java-sdk-s3:jar:1.11.45:compile
          [INFO] |  +- com.amazonaws:aws-java-sdk-kms:jar:1.11.45:compile
          [INFO] |  +- com.amazonaws:aws-java-sdk-core:jar:1.11.45:compile
          [INFO] |  |  \- software.amazon.ion:ion-java:jar:1.0.1:compile
          [INFO] |  \- com.amazonaws:jmespath-java:jar:1.0:compile
          [INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.6.6:compile
          [INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.6:compile
          [INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.6:compile
          [INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.6:compile
          [INFO] +- joda-time:joda-time:jar:2.9.4:compile
          [INFO] +- com.amazonaws:aws-java-sdk-sts:jar:1.11.45:test
          [INFO] +- junit:junit:jar:4.11:test
          [INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
          [INFO] +- org.mockito:mockito-all:jar:1.8.5:test
          [INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.9.0-SNAPSHOT:test
          [INFO] |  +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  +- org.apache.hadoop:hadoop-yarn-client:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  \- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.9.0-SNAPSHOT:test
          [INFO] |  +- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:test
          [INFO] |  \- com.google.inject.extensions:guice-servlet:jar:3.0:test
          [INFO] |     \- com.google.inject:guice:jar:3.0:test
          [INFO] |        +- javax.inject:javax.inject:jar:1:test
          [INFO] |        \- aopalliance:aopalliance:jar:1.0:test
          [INFO] +- org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:2.9.0-SNAPSHOT:test
          [INFO] |  +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.9.0-SNAPSHOT:test
          [INFO] |  +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  +- com.sun.jersey:jersey-client:jar:1.9:test
          [INFO] |  |  +- com.codahale.metrics:metrics-core:jar:3.0.1:test
          [INFO] |  |  \- com.sun.jersey.contribs:jersey-guice:jar:1.9:test
          [INFO] |  +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  |  \- de.ruedigermoeller:fst:jar:2.24:test
          [INFO] |  |  |     \- org.objenesis:objenesis:jar:2.1:test
          [INFO] |  |  +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.9.0-SNAPSHOT:test
          [INFO] |  |  +- org.apache.curator:curator-test:jar:2.7.1:test
          [INFO] |  |  |  +- org.javassist:javassist:jar:3.18.1-GA:test
          [INFO] |  |  |  \- org.apache.commons:commons-math:jar:2.2:test
          [INFO] |  |  \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:test
          [INFO] |  +- org.apache.hadoop:hadoop-yarn-common:jar:2.9.0-SNAPSHOT:test
          [INFO] |  \- org.apache.hadoop:hadoop-yarn-api:jar:2.9.0-SNAPSHOT:test
          [INFO] +- org.apache.hadoop:hadoop-mapreduce-examples:jar:2.9.0-SNAPSHOT:test
          [INFO] |  \- org.apache.hadoop:hadoop-hdfs-client:jar:2.9.0-SNAPSHOT:test
          [INFO] |     \- com.squareup.okhttp:okhttp:jar:2.4.0:test
          [INFO] |        \- com.squareup.okio:okio:jar:1.4.0:test
          [INFO] +- org.apache.hadoop:hadoop-distcp:jar:2.9.0-SNAPSHOT:test
          [INFO] \- org.apache.hadoop:hadoop-distcp:test-jar:tests:2.9.0-SNAPSHOT:test
          
          Show
          cnauroth Chris Nauroth added a comment - This needed a rebase after one of my patches, so I went ahead and updated it. I'm attaching revision 002. This time, I went to AWS SDK 1.11.45. All tests passed against US-west-2. Here is the updated dependency tree: [INFO] --- maven-dependency-plugin:2.2:tree ( default -cli) @ hadoop-aws --- [INFO] org.apache.hadoop:hadoop-aws:jar:2.9.0-SNAPSHOT [INFO] +- org.apache.hadoop:hadoop-common:jar:2.9.0-SNAPSHOT:compile [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.9.0-SNAPSHOT:compile [INFO] | | \- jdk.tools:jdk.tools:jar:1.7:system [INFO] | +- com.google.guava:guava:jar:11.0.2:compile [INFO] | +- commons-cli:commons-cli:jar:1.2:compile [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile [INFO] | +- xmlenc:xmlenc:jar:0.52:compile [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile [INFO] | +- commons-codec:commons-codec:jar:1.4:compile [INFO] | +- commons-io:commons-io:jar:2.4:compile [INFO] | +- commons-net:commons-net:jar:3.1:compile [INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile [INFO] | +- javax.servlet:servlet-api:jar:2.5:compile [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile [INFO] | +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26:compile [INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime [INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile [INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile [INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile [INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile [INFO] | | | \- javax.activation:activation:jar:1.1:compile [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile (version managed from 1.8.3) [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile (version managed from 1.8.3) [INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile [INFO] | | \- asm:asm:jar:3.2:compile (version managed from 3.1) [INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile [INFO] | +- log4j:log4j:jar:1.2.17:compile [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:compile [INFO] | | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile [INFO] | +- commons-lang:commons-lang:jar:2.6:compile [INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile [INFO] | | +- commons-digester:commons-digester:jar:1.8:compile [INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile [INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile [INFO] | +- org.slf4j:slf4j-api:jar:1.7.10:compile [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | +- org.apache.avro:avro:jar:1.7.4:compile [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile [INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile [INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile [INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.9.0-SNAPSHOT:compile [INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile [INFO] | | | +- net.jcip:jcip-annotations:jar:1.0:compile [INFO] | | | \- net.minidev:json-smart:jar:1.1.1:compile [INFO] | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile [INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile [INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile [INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile [INFO] | | \- org.apache.curator:curator-framework:jar:2.7.1:compile [INFO] | +- com.jcraft:jsch:jar:0.1.51:compile [INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile [INFO] | +- org.apache.curator:curator-recipes:jar:2.7.1:compile [INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.0:compile [INFO] | +- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile [INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile [INFO] | | +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile (version managed from 1.6.1) [INFO] | | \- io.netty:netty:jar:3.6.2.Final:compile (version managed from 3.7.0.Final) [INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile [INFO] | \- org.tukaani:xz:jar:1.0:compile [INFO] +- org.apache.hadoop:hadoop-common:test-jar:tests:2.9.0-SNAPSHOT:test [INFO] +- com.amazonaws:aws-java-sdk-s3:jar:1.11.45:compile [INFO] | +- com.amazonaws:aws-java-sdk-kms:jar:1.11.45:compile [INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.11.45:compile [INFO] | | \- software.amazon.ion:ion-java:jar:1.0.1:compile [INFO] | \- com.amazonaws:jmespath-java:jar:1.0:compile [INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.6.6:compile [INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.6:compile [INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.6:compile [INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.6:compile [INFO] +- joda-time:joda-time:jar:2.9.4:compile [INFO] +- com.amazonaws:aws-java-sdk-sts:jar:1.11.45:test [INFO] +- junit:junit:jar:4.11:test [INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test [INFO] +- org.mockito:mockito-all:jar:1.8.5:test [INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.9.0-SNAPSHOT:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.9.0-SNAPSHOT:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.9.0-SNAPSHOT:test [INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.9.0-SNAPSHOT:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.9.0-SNAPSHOT:test [INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:test [INFO] | \- com.google.inject.extensions:guice-servlet:jar:3.0:test [INFO] | \- com.google.inject:guice:jar:3.0:test [INFO] | +- javax.inject:javax.inject:jar:1:test [INFO] | \- aopalliance:aopalliance:jar:1.0:test [INFO] +- org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:2.9.0-SNAPSHOT:test [INFO] | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.9.0-SNAPSHOT:test [INFO] | +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.9.0-SNAPSHOT:test [INFO] | | +- com.sun.jersey:jersey-client:jar:1.9:test [INFO] | | +- com.codahale.metrics:metrics-core:jar:3.0.1:test [INFO] | | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:test [INFO] | +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.9.0-SNAPSHOT:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.9.0-SNAPSHOT:test [INFO] | | | \- de.ruedigermoeller:fst:jar:2.24:test [INFO] | | | \- org.objenesis:objenesis:jar:2.1:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.9.0-SNAPSHOT:test [INFO] | | +- org.apache.curator:curator-test:jar:2.7.1:test [INFO] | | | +- org.javassist:javassist:jar:3.18.1-GA:test [INFO] | | | \- org.apache.commons:commons-math:jar:2.2:test [INFO] | | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:test [INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:2.9.0-SNAPSHOT:test [INFO] | \- org.apache.hadoop:hadoop-yarn-api:jar:2.9.0-SNAPSHOT:test [INFO] +- org.apache.hadoop:hadoop-mapreduce-examples:jar:2.9.0-SNAPSHOT:test [INFO] | \- org.apache.hadoop:hadoop-hdfs-client:jar:2.9.0-SNAPSHOT:test [INFO] | \- com.squareup.okhttp:okhttp:jar:2.4.0:test [INFO] | \- com.squareup.okio:okio:jar:1.4.0:test [INFO] +- org.apache.hadoop:hadoop-distcp:jar:2.9.0-SNAPSHOT:test [INFO] \- org.apache.hadoop:hadoop-distcp:test-jar:tests:2.9.0-SNAPSHOT:test
          Hide
          cnauroth Chris Nauroth added a comment -

          Also, AWS SDK 1.11.45 depends on Jackson 2.6.6.

          Show
          cnauroth Chris Nauroth added a comment - Also, AWS SDK 1.11.45 depends on Jackson 2.6.6.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 53s Maven dependency ordering for branch
          +1 mvninstall 6m 45s branch-2 passed
          +1 compile 5m 36s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 30s branch-2 passed with JDK v1.7.0_111
          +1 mvnsite 0m 35s branch-2 passed
          +1 mvneclipse 0m 23s branch-2 passed
          +1 javadoc 0m 21s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 23s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 23s Maven dependency ordering for patch
          +1 mvninstall 0m 25s the patch passed
          +1 compile 5m 51s the patch passed with JDK v1.8.0_101
          -1 javac 5m 51s root-jdk1.8.0_101 with JDK v1.8.0_101 generated 7 new + 857 unchanged - 0 fixed = 864 total (was 857)
          +1 compile 6m 52s the patch passed with JDK v1.7.0_111
          -1 javac 6m 52s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 7 new + 950 unchanged - 0 fixed = 957 total (was 950)
          +1 mvnsite 0m 37s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 22s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_111
          +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 24s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          39m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13050
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834265/HADOOP-13050-branch-2.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 78bacacd5bd3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ad7d3c4
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 53s Maven dependency ordering for branch +1 mvninstall 6m 45s branch-2 passed +1 compile 5m 36s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 30s branch-2 passed with JDK v1.7.0_111 +1 mvnsite 0m 35s branch-2 passed +1 mvneclipse 0m 23s branch-2 passed +1 javadoc 0m 21s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 23s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 23s Maven dependency ordering for patch +1 mvninstall 0m 25s the patch passed +1 compile 5m 51s the patch passed with JDK v1.8.0_101 -1 javac 5m 51s root-jdk1.8.0_101 with JDK v1.8.0_101 generated 7 new + 857 unchanged - 0 fixed = 864 total (was 857) +1 compile 6m 52s the patch passed with JDK v1.7.0_111 -1 javac 6m 52s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 7 new + 950 unchanged - 0 fixed = 957 total (was 950) +1 mvnsite 0m 37s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 22s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 26s the patch passed with JDK v1.7.0_111 +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 0m 24s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 39m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13050 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834265/HADOOP-13050-branch-2.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 78bacacd5bd3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ad7d3c4 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10831/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm uploading revision 003, which fixes new deprecation warnings triggered by the SDK upgrade.

          Show
          cnauroth Chris Nauroth added a comment - I'm uploading revision 003, which fixes new deprecation warnings triggered by the SDK upgrade.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 54s Maven dependency ordering for branch
          +1 mvninstall 6m 36s branch-2 passed
          +1 compile 5m 38s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 41s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 1m 26s branch-2 passed
          +1 mvnsite 0m 38s branch-2 passed
          +1 mvneclipse 0m 28s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 38s branch-2 passed
          +1 javadoc 0m 26s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 31s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 26s Maven dependency ordering for patch
          +1 mvninstall 0m 31s the patch passed
          +1 compile 5m 37s the patch passed with JDK v1.8.0_101
          +1 javac 5m 37s the patch passed
          +1 compile 6m 48s the patch passed with JDK v1.7.0_111
          +1 javac 6m 48s the patch passed
          +1 checkstyle 1m 29s the patch passed
          +1 mvnsite 0m 43s the patch passed
          +1 mvneclipse 0m 37s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 51s the patch passed
          +1 javadoc 0m 33s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 38s the patch passed with JDK v1.7.0_111
          +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          68m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13050
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834277/HADOOP-13050-branch-2.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux e847a6a0350c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ad7d3c4
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10833/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10833/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 54s Maven dependency ordering for branch +1 mvninstall 6m 36s branch-2 passed +1 compile 5m 38s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 41s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 1m 26s branch-2 passed +1 mvnsite 0m 38s branch-2 passed +1 mvneclipse 0m 28s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 38s branch-2 passed +1 javadoc 0m 26s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 31s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 26s Maven dependency ordering for patch +1 mvninstall 0m 31s the patch passed +1 compile 5m 37s the patch passed with JDK v1.8.0_101 +1 javac 5m 37s the patch passed +1 compile 6m 48s the patch passed with JDK v1.7.0_111 +1 javac 6m 48s the patch passed +1 checkstyle 1m 29s the patch passed +1 mvnsite 0m 43s the patch passed +1 mvneclipse 0m 37s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 51s the patch passed +1 javadoc 0m 33s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 38s the patch passed with JDK v1.7.0_111 +1 unit 0m 16s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 0m 28s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 68m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13050 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834277/HADOOP-13050-branch-2.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux e847a6a0350c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ad7d3c4 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10833/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10833/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Jackson update should go under HADOOP-12705; see discussion there on versions

          Show
          stevel@apache.org Steve Loughran added a comment - Jackson update should go under HADOOP-12705 ; see discussion there on versions
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Raised the priority to blocker because of the license issue reported by HADOOP-13794.

          Show
          ajisakaa Akira Ajisaka added a comment - Raised the priority to blocker because of the license issue reported by HADOOP-13794 .
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This is out of sync with HADOOP-13727; cancelling the patch with the intent of producing a resynced version

          Show
          stevel@apache.org Steve Loughran added a comment - This is out of sync with HADOOP-13727 ; cancelling the patch with the intent of producing a resynced version
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 003 increments the AWS SDK and puts jackson to 2.7.8, as discussed elsewhere. The real jackson update needs to be a self contained JIRA/patch/commit; this patch exists to show that the two libraries coexist

          tested against s3a ireland, everything working except ITestS3AMiniYarnCluster. This is covered by HADOOP-13081; this patch (temporarily) includes more detail on the assert failure; here's where I saw it first.

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 003 increments the AWS SDK and puts jackson to 2.7.8, as discussed elsewhere. The real jackson update needs to be a self contained JIRA/patch/commit; this patch exists to show that the two libraries coexist tested against s3a ireland, everything working except ITestS3AMiniYarnCluster . This is covered by HADOOP-13081 ; this patch (temporarily) includes more detail on the assert failure; here's where I saw it first.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 1m 1s Maven dependency ordering for branch
          -1 mvninstall 3m 10s root in branch-2 failed.
          -1 compile 1m 35s root in branch-2 failed with JDK v1.8.0_101.
          -1 compile 1m 44s root in branch-2 failed with JDK v1.7.0_111.
          +1 checkstyle 1m 30s branch-2 passed
          +1 mvnsite 0m 37s branch-2 passed
          +1 mvneclipse 0m 23s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 35s branch-2 passed
          +1 javadoc 0m 24s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 28s branch-2 passed with JDK v1.7.0_111
          0 mvndep 1m 21s Maven dependency ordering for patch
          +1 mvninstall 0m 31s the patch passed
          -1 compile 1m 30s root in the patch failed with JDK v1.8.0_101.
          -1 javac 1m 30s root in the patch failed with JDK v1.8.0_101.
          -1 compile 1m 44s root in the patch failed with JDK v1.7.0_111.
          -1 javac 1m 44s root in the patch failed with JDK v1.7.0_111.
          +1 checkstyle 1m 26s the patch passed
          +1 mvnsite 0m 41s the patch passed
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 50s the patch passed
          +1 javadoc 0m 29s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 31s the patch passed with JDK v1.7.0_111
          +1 unit 0m 14s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 27s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          48m 4s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13050
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837817/HADOOP-13050-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux f1f541367efd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / a942d96
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-mvninstall-root.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-compile-root-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-compile-root-jdk1.7.0_111.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 1s Maven dependency ordering for branch -1 mvninstall 3m 10s root in branch-2 failed. -1 compile 1m 35s root in branch-2 failed with JDK v1.8.0_101. -1 compile 1m 44s root in branch-2 failed with JDK v1.7.0_111. +1 checkstyle 1m 30s branch-2 passed +1 mvnsite 0m 37s branch-2 passed +1 mvneclipse 0m 23s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 35s branch-2 passed +1 javadoc 0m 24s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 28s branch-2 passed with JDK v1.7.0_111 0 mvndep 1m 21s Maven dependency ordering for patch +1 mvninstall 0m 31s the patch passed -1 compile 1m 30s root in the patch failed with JDK v1.8.0_101. -1 javac 1m 30s root in the patch failed with JDK v1.8.0_101. -1 compile 1m 44s root in the patch failed with JDK v1.7.0_111. -1 javac 1m 44s root in the patch failed with JDK v1.7.0_111. +1 checkstyle 1m 26s the patch passed +1 mvnsite 0m 41s the patch passed +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 50s the patch passed +1 javadoc 0m 29s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 31s the patch passed with JDK v1.7.0_111 +1 unit 0m 14s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 0m 27s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 48m 4s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13050 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837817/HADOOP-13050-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux f1f541367efd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / a942d96 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-compile-root-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/branch-compile-root-jdk1.7.0_111.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.8.0_101.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/artifact/patchprocess/patch-compile-root-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11019/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          yetus failure due to branch-2 not compiling; assume unrelated. I'll resubmit once fixed.

          I'm considering if we could wrap the AWS SDK which something that shades Jackson, that is — create an intermediate module whose purpose is to isolate the AWS SDK jackson and so make using this release in 2.7.x straightforward, with branch-2 choosing whatever version of jackson people want. It'd be back to being some uber-AWS JAR. Where there's a risk here is if someone wants to use an AWS library which we don't bundle (say: kinesis, SNS, ...) and they get linked up, or worse: include duplicate dependencies.

          I am so looking forward to Java 9, even if I have no idea how we'd get Hadoop to work there.

          Show
          stevel@apache.org Steve Loughran added a comment - yetus failure due to branch-2 not compiling; assume unrelated. I'll resubmit once fixed. I'm considering if we could wrap the AWS SDK which something that shades Jackson, that is — create an intermediate module whose purpose is to isolate the AWS SDK jackson and so make using this release in 2.7.x straightforward, with branch-2 choosing whatever version of jackson people want. It'd be back to being some uber-AWS JAR. Where there's a risk here is if someone wants to use an AWS library which we don't bundle (say: kinesis, SNS, ...) and they get linked up, or worse: include duplicate dependencies. I am so looking forward to Java 9, even if I have no idea how we'd get Hadoop to work there.
          Hide
          liuml07 Mingliang Liu added a comment -

          Rebased the v2 patch from trunk which has the Jackson version upgraded along with its dependency jars for hadoop-aws modules explicitly declared. The v2 patch also borrows the code from branch-2 patch that address some of the deprecated S3 methods.

          Ping Steve Loughran for review. You may have to +1 for your own patch. Ping Akira Ajisaka as well.

          Shading Jackson seems very helpful. I'm happy to review if we have such; before that we can get this Blocker in first.

          Show
          liuml07 Mingliang Liu added a comment - Rebased the v2 patch from trunk which has the Jackson version upgraded along with its dependency jars for hadoop-aws modules explicitly declared. The v2 patch also borrows the code from branch-2 patch that address some of the deprecated S3 methods. Ping Steve Loughran for review. You may have to +1 for your own patch. Ping Akira Ajisaka as well. Shading Jackson seems very helpful. I'm happy to review if we have such; before that we can get this Blocker in first.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 18s Maven dependency ordering for branch
          +1 mvninstall 8m 46s trunk passed
          +1 compile 10m 42s trunk passed
          +1 checkstyle 1m 36s trunk passed
          +1 mvnsite 0m 47s trunk passed
          +1 mvneclipse 0m 42s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 36s trunk passed
          +1 javadoc 0m 36s trunk passed
          0 mvndep 0m 29s Maven dependency ordering for patch
          +1 mvninstall 0m 31s the patch passed
          +1 compile 9m 17s the patch passed
          -1 javac 9m 17s root generated 1 new + 713 unchanged - 0 fixed = 714 total (was 713)
          +1 checkstyle 1m 35s the patch passed
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 44s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 45s the patch passed
          +1 javadoc 0m 37s the patch passed
          +1 unit 0m 15s hadoop-project in the patch passed.
          +1 unit 0m 36s hadoop-aws in the patch passed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          63m 24s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13050
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839930/HADOOP-13050-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux f4261b0fb2ea 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6f80742
          Default Java 1.8.0_111
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/artifact/patchprocess/diff-compile-javac-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 8m 46s trunk passed +1 compile 10m 42s trunk passed +1 checkstyle 1m 36s trunk passed +1 mvnsite 0m 47s trunk passed +1 mvneclipse 0m 42s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 36s trunk passed +1 javadoc 0m 36s trunk passed 0 mvndep 0m 29s Maven dependency ordering for patch +1 mvninstall 0m 31s the patch passed +1 compile 9m 17s the patch passed -1 javac 9m 17s root generated 1 new + 713 unchanged - 0 fixed = 714 total (was 713) +1 checkstyle 1m 35s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 45s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 0m 15s hadoop-project in the patch passed. +1 unit 0m 36s hadoop-aws in the patch passed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 63m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13050 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839930/HADOOP-13050-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux f4261b0fb2ea 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6f80742 Default Java 1.8.0_111 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/artifact/patchprocess/diff-compile-javac-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11112/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 004; expects jackson to be already upgraded. tests work, though I got a root-dir-not-empty failure on {{ITestS3AContractRootDir}] which went away when run standalone.

          Show
          stevel@apache.org Steve Loughran added a comment - patch 004; expects jackson to be already upgraded. tests work, though I got a root-dir-not-empty failure on {{ITestS3AContractRootDir}] which went away when run standalone.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          oops, I did the rebase/resubmit without noticing yours; will D/L it and apply. FWIW, all source code changes are actually Chris's; all my patch did was update the JAR versions & add the dependency

          Show
          stevel@apache.org Steve Loughran added a comment - oops, I did the rebase/resubmit without noticing yours; will D/L it and apply. FWIW, all source code changes are actually Chris's; all my patch did was update the JAR versions & add the dependency
          Hide
          liuml07 Mingliang Liu added a comment -

          Steve Loughran no worries. The target branch is both branch-2 and tunk so we have to provide a similar branch-2 patch as well. Fortunately our patches are exactly the same with just one line of new break. So I guess you just +1 on this patch? If we should thank Chris for original patch, should I use "contributed by Chris and Steve" when committing?

          One last concern from the Jenkins run is that, HADOOP-13727 added the SharedInstanceProfileCredentialsProvider, which can be removed after this AWS SDK upgrade. Do you think we address it here, or separately?
          Chris also mentioned this in the JIRA:

          As of AWS SDK 1.11.39, the SDK code internally enforces a singleton. After Hadoop upgrades to that version or higher, it's likely that we can remove this code.

          Show
          liuml07 Mingliang Liu added a comment - Steve Loughran no worries. The target branch is both branch-2 and tunk so we have to provide a similar branch-2 patch as well. Fortunately our patches are exactly the same with just one line of new break. So I guess you just +1 on this patch? If we should thank Chris for original patch, should I use "contributed by Chris and Steve" when committing? One last concern from the Jenkins run is that, HADOOP-13727 added the SharedInstanceProfileCredentialsProvider , which can be removed after this AWS SDK upgrade. Do you think we address it here, or separately? Chris also mentioned this in the JIRA: As of AWS SDK 1.11.39, the SDK code internally enforces a singleton. After Hadoop upgrades to that version or higher, it's likely that we can remove this code.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Just finished testing spark & s3 integration with this; all is well

          Show
          stevel@apache.org Steve Loughran added a comment - +1 Just finished testing spark & s3 integration with this; all is well
          Hide
          cnauroth Chris Nauroth added a comment -

          Mingliang Liu, thank you for reviewing.

          One last concern from the Jenkins run is that, HADOOP-13727 added the SharedInstanceProfileCredentialsProvider, which can be removed after this AWS SDK upgrade. Do you think we address it here, or separately?

          I'd suggest handling it in a separate JIRA, targeted only to 3.x (not 2.x). It's possible that users of 2.8 will start referring to SharedInstanceProfileCredentialsProvider in their configuration. If it were removed in 2.9, then those configurations would break after a user upgrades from 2.8 to 2.9. Waiting until 3.x for that kind of change would adhere to our compatibility guidelines and give those users more of a warning.

          Show
          cnauroth Chris Nauroth added a comment - Mingliang Liu , thank you for reviewing. One last concern from the Jenkins run is that, HADOOP-13727 added the SharedInstanceProfileCredentialsProvider, which can be removed after this AWS SDK upgrade. Do you think we address it here, or separately? I'd suggest handling it in a separate JIRA, targeted only to 3.x (not 2.x). It's possible that users of 2.8 will start referring to SharedInstanceProfileCredentialsProvider in their configuration. If it were removed in 2.9, then those configurations would break after a user upgrades from 2.8 to 2.9. Waiting until 3.x for that kind of change would adhere to our compatibility guidelines and give those users more of a warning.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Chris Nauroth!

          That makes perfect sense address this for trunk branch separately; and marking as incompatible change.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Chris Nauroth ! That makes perfect sense address this for trunk branch separately; and marking as incompatible change.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 1m 16s Maven dependency ordering for branch
          +1 mvninstall 6m 34s branch-2 passed
          +1 compile 5m 55s branch-2 passed with JDK v1.8.0_111
          +1 compile 7m 8s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 2m 23s branch-2 passed
          +1 mvnsite 1m 6s branch-2 passed
          +1 mvneclipse 0m 51s branch-2 passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 4s branch-2 passed
          +1 javadoc 0m 57s branch-2 passed with JDK v1.8.0_111
          +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111
          0 mvndep 0m 31s Maven dependency ordering for patch
          +1 mvninstall 0m 51s the patch passed
          +1 compile 7m 11s the patch passed with JDK v1.8.0_111
          -1 javac 7m 11s root-jdk1.8.0_111 with JDK v1.8.0_111 generated 1 new + 854 unchanged - 0 fixed = 855 total (was 854)
          +1 compile 6m 52s the patch passed with JDK v1.7.0_111
          -1 javac 6m 52s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 947 unchanged - 0 fixed = 948 total (was 947)
          +1 checkstyle 1m 29s the patch passed
          +1 mvnsite 0m 40s the patch passed
          +1 mvneclipse 0m 35s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 0m 50s the patch passed
          +1 javadoc 0m 32s the patch passed with JDK v1.8.0_111
          +1 javadoc 0m 38s the patch passed with JDK v1.7.0_111
          +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111.
          +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          75m 24s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13050
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840045/HADOOP-13050-branch-2-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 31254f540383 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 06f5d27
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 16s Maven dependency ordering for branch +1 mvninstall 6m 34s branch-2 passed +1 compile 5m 55s branch-2 passed with JDK v1.8.0_111 +1 compile 7m 8s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 2m 23s branch-2 passed +1 mvnsite 1m 6s branch-2 passed +1 mvneclipse 0m 51s branch-2 passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 4s branch-2 passed +1 javadoc 0m 57s branch-2 passed with JDK v1.8.0_111 +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111 0 mvndep 0m 31s Maven dependency ordering for patch +1 mvninstall 0m 51s the patch passed +1 compile 7m 11s the patch passed with JDK v1.8.0_111 -1 javac 7m 11s root-jdk1.8.0_111 with JDK v1.8.0_111 generated 1 new + 854 unchanged - 0 fixed = 855 total (was 854) +1 compile 6m 52s the patch passed with JDK v1.7.0_111 -1 javac 6m 52s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 947 unchanged - 0 fixed = 948 total (was 947) +1 checkstyle 1m 29s the patch passed +1 mvnsite 0m 40s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 0m 50s the patch passed +1 javadoc 0m 32s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 38s the patch passed with JDK v1.7.0_111 +1 unit 0m 15s hadoop-project in the patch passed with JDK v1.7.0_111. +1 unit 0m 29s hadoop-aws in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 75m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13050 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840045/HADOOP-13050-branch-2-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 31254f540383 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 06f5d27 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_111.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11115/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed this to trunk and branch-2 branches.

          Thanks Chris Nauroth and Steve Loughran for your contribution. This will benefit us a lot especially for S3Guard, singleton InstanceProfileCredentialsProvider, faster listing.. to name a few.

          Show
          liuml07 Mingliang Liu added a comment - Committed this to trunk and branch-2 branches. Thanks Chris Nauroth and Steve Loughran for your contribution. This will benefit us a lot especially for S3Guard, singleton InstanceProfileCredentialsProvider, faster listing.. to name a few.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10876 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10876/)
          HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth (liuml07: rev af791b774bdc3af26ab99487ffea7246c6788a9c)

          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3AMiniYarnCluster.java
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java
          • (edit) hadoop-project/pom.xml
          • (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java
          • (edit) hadoop-tools/hadoop-aws/pom.xml
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10876 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10876/ ) HADOOP-13050 . Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth (liuml07: rev af791b774bdc3af26ab99487ffea7246c6788a9c) (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/yarn/ITestS3AMiniYarnCluster.java (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java (edit) hadoop-project/pom.xml (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java (edit) hadoop-tools/hadoop-aws/pom.xml (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development