Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8984

S3 File Permissions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Incomplete
    • 2.0.2-alpha
    • None
    • None
    • None
    • s3filesystem permissions owner staging

    Description

      Till lately I've been using 0.20.2 and everything was ok. Now I'm using the latest trunc 0.22.0-SNAPSHOT and getting the following thrown:

      Exception in thread "main" java.io.IOException: The ownership/permissions on the staging directory s3://my-s3-bucket/mnt/hadoop.tmp.dir/mapred/staging/root/.staging is not as expected. It is owned by and permissions are rwxrwxrwx. The directory must be owned by the submitter root or by root and permissions must be rwx------
      at
      org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:107)
      at
      org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:312)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:961)
      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:977)
      at com.mycompany.MyJob.runJob(MyJob.java:153)
      at com.mycompany.MyJob.run(MyJob.java:177)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at com.mycompany.MyOtherJob.runJob(MyOtherJob.java:62)
      at com.mycompany.MyOtherJob.run(MyOtherJob.java:112)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at com.mycompany.MyOtherJob.main(MyOtherJob.java:117)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:187)

      (The "it is owned by ... and permissions " is not a mistake, seems like the empty string is printed there)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dleshem Danny Leshem
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: