Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4063

Missing files/classes needed for S3a access

Attach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Specifying

      "connection": "s3a://<bucketname>"
      

      results in the following error:

      Error: SYSTEM ERROR: ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
      

      I can fix this by dropping in these files from the hadoop binary tarball:
      hadoop-aws-2.6.2.jar
      aws-java-sdk-1.7.4.jar

      And then adding this to my core-site.xml:

        <property>
          <name>fs.s3a.access.key</name>
          <value>ACCESSKEY</value>
        </property>
      
        <property>
          <name>fs.s3a.secret.key</name>
          <value>SECRETKEY</value>
        </property>
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            apol Abhijit Pol
            ngriffith Nathan Griffith
            Votes:
            0 Vote for this issue
            Watchers:
            3 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment