Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12100

ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.1, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission.

      This bug is currently causing the TestRollingLevelDBTimelineStore test to fail. HADOOP-11347 changed some code to use applyUmask, which exposed this bug.

      Running org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore
      Tests run: 16, Failures: 0, Errors: 16, Skipped: 0, Time elapsed: 2.65 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore
      testGetDomains(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 1.533 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testRelatingToNonExistingEntity(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.085 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testValidateConfig(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.07 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEntitiesWithPrimaryFilters(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.061 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testRelatingToOldEntityWithoutDomainId(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.055 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testRootDirPermission(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.056 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetSingleEntity(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.05 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testCacheSizes(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.051 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEntities(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.049 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testRelatingToEntityInSamePut(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.056 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetDomain(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.049 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEvents(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.049 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEntitiesWithSecondaryFilters(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.05 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testCheckVersion(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.05 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEntitiesWithFromId(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.055 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      testGetEntitiesWithFromTs(org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore)  Time elapsed: 0.049 sec  <<< ERROR!
      java.lang.UnsupportedOperationException: null
      	at org.apache.hadoop.fs.permission.FsPermission$ImmutableFsPermission.applyUMask(FsPermission.java:380)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:496)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:551)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:529)
      	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:314)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.initFileSystem(RollingLevelDB.java:207)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDB.init(RollingLevelDB.java:200)
      	at org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore.setup(TestRollingLevelDBTimelineStore.java:65)
      
      
      Results :
      
      Tests in error: 
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
        TestRollingLevelDBTimelineStore.setup:65 » UnsupportedOperation
      
      Tests run: 16, Failures: 0, Errors: 16, Skipped: 0
      
      1. 0001-YARN-3812.patch
        1 kB
        Bibin A Chundatt
      2. 0002-YARN-3812.patch
        0.9 kB
        Bibin A Chundatt

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #222 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/222/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #222 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/222/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2161 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2161/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2161 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2161/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #963 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/963/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #963 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/963/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #233 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/233/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #233 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/233/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2178/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2178/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8038 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8038/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8038 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8038/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/230/)
          HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/230/ ) HADOOP-12100 . ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) (cmccabe: rev 6e0a9f92fe0052d39b95a605174b3f6423c6aae7) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          cmccabe Colin P. McCabe added a comment -

          +1. Thanks

          Show
          cmccabe Colin P. McCabe added a comment - +1. Thanks
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 19s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 3s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 2s Tests passed in hadoop-common.
              60m 57s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12740269/0002-YARN-3812.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2ad6687
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 19s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 3s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 2s Tests passed in hadoop-common.     60m 57s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740269/0002-YARN-3812.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2ad6687 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6989/console This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Colin P. McCabe and Jason Lowe Have uploaded the patch and have moved the same to common since the fix is on common.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Colin P. McCabe and Jason Lowe Have uploaded the patch and have moved the same to common since the fix is on common.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Thnk you for review comments .As per comments have updated ImmutableFspermission .Please review the patch. TestRollingLevelDB class all test are passing.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Thnk you for review comments .As per comments have updated ImmutableFspermission .Please review the patch. TestRollingLevelDB class all test are passing.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Yes, ImmutableFsPermission should not be overriding applyUMask since the method does not actually mutate the object. readFields does mutate and therefore is appropriate for preventing invocation for constant objects.

          I agree. It seems that FsPermission#ImmutablePermission is incorrectly overriding FsPermission#applyUMask. There is no reason to override this method since it doesn't modify the FsPermission. The right fix should be to simply stop overriding that method. Do you want to move the JIRA over to Hadoop-common and post a patch for that?

          Show
          cmccabe Colin P. McCabe added a comment - Yes, ImmutableFsPermission should not be overriding applyUMask since the method does not actually mutate the object. readFields does mutate and therefore is appropriate for preventing invocation for constant objects. I agree. It seems that FsPermission#ImmutablePermission is incorrectly overriding FsPermission#applyUMask . There is no reason to override this method since it doesn't modify the FsPermission . The right fix should be to simply stop overriding that method. Do you want to move the JIRA over to Hadoop-common and post a patch for that?
          Hide
          jlowe Jason Lowe added a comment -

          Yes, ImmutableFsPermission should not be overriding applyUMask since the method does not actually mutate the object. readFields does mutate and therefore is appropriate for preventing invocation for constant objects.

          The main point is that we shouldn't be blaming the client code and say it was wrong to pass a constant permission object. If the filesystem can't handle constant permissions being passed to it then it's a bug in the framework not the client code.

          Show
          jlowe Jason Lowe added a comment - Yes, ImmutableFsPermission should not be overriding applyUMask since the method does not actually mutate the object. readFields does mutate and therefore is appropriate for preventing invocation for constant objects. The main point is that we shouldn't be blaming the client code and say it was wrong to pass a constant permission object. If the filesystem can't handle constant permissions being passed to it then it's a bug in the framework not the client code.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Jason Lowe Thnks for the review. Currently the ImmutableFsPermission#applyUMask is unsupported and was causing the failure. So as i understand the you would like the have implementation change for applyUmask in ImmutableFsPermission

            private static class ImmutableFsPermission extends FsPermission {
              public ImmutableFsPermission(short permission) {
                super(permission);
              }
              @Override
              public FsPermission applyUMask(FsPermission umask) {
                throw new UnsupportedOperationException();
              }
              @Override
              public void readFields(DataInput in) throws IOException {
                throw new UnsupportedOperationException();
              }    
            }
          }
          

          removed the overriding of the same ? Any suggestion ?
          Also Varun Saxena any comments one the same?

          Show
          bibinchundatt Bibin A Chundatt added a comment - Jason Lowe Thnks for the review. Currently the ImmutableFsPermission#applyUMask is unsupported and was causing the failure. So as i understand the you would like the have implementation change for applyUmask in ImmutableFsPermission private static class ImmutableFsPermission extends FsPermission { public ImmutableFsPermission( short permission) { super (permission); } @Override public FsPermission applyUMask(FsPermission umask) { throw new UnsupportedOperationException(); } @Override public void readFields(DataInput in) throws IOException { throw new UnsupportedOperationException(); } } } removed the overriding of the same ? Any suggestion ? Also Varun Saxena any comments one the same?
          Hide
          jlowe Jason Lowe added a comment -

          I don't think this is the correct fix. RollingLevelDBTimelineStore is using an immutable permission to specify the umask. If the proposed fix is to make it mutable, then that implies we're allowing a constant to be modified by the filesystem. If this permission were passed to multiple subsystems, having it magically mutate by one subsystem using it is probably not the correct approach.

          Instead it appears to be a problem with ImmutableFsPermission. It prevents applyUmask from being invoked, but in reality applyUmask does not modify the permission object – it returns a new permission object instead. Therefore I think the real bug is that ImmutableFsPermission is preventing applyUmask from being invoked.

          Show
          jlowe Jason Lowe added a comment - I don't think this is the correct fix. RollingLevelDBTimelineStore is using an immutable permission to specify the umask. If the proposed fix is to make it mutable, then that implies we're allowing a constant to be modified by the filesystem. If this permission were passed to multiple subsystems, having it magically mutate by one subsystem using it is probably not the correct approach. Instead it appears to be a problem with ImmutableFsPermission. It prevents applyUmask from being invoked, but in reality applyUmask does not modify the permission object – it returns a new permission object instead. Therefore I think the real bug is that ImmutableFsPermission is preventing applyUmask from being invoked.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Jenkins not triggering so cancelled and submitted patch again

          Show
          bibinchundatt Bibin A Chundatt added a comment - Jenkins not triggering so cancelled and submitted patch again
          Hide
          rkanter Robert Kanter added a comment -

          +1 pending Jenkins

          Show
          rkanter Robert Kanter added a comment - +1 pending Jenkins
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Please review patch uploaded

          Show
          bibinchundatt Bibin A Chundatt added a comment - Please review patch uploaded
          Hide
          rkanter Robert Kanter added a comment -

          Bibin A Chundatt, good find! That fixes the test. Would you like to assign this JIRA to yourself and post a patch with the change?

          Show
          rkanter Robert Kanter added a comment - Bibin A Chundatt , good find! That fixes the test. Would you like to assign this JIRA to yourself and post a patch with the change?
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          RawLocalFileSystem#mkOneDirWithMode applyUMask is failing. In RollingLevelDBTimelineStore

            static final FsPermission LEVELDB_DIR_UMASK = FsPermission
                .createImmutable((short) 0700);
          

          change to

          static final FsPermission LEVELDB_DIR_UMASK =  new FsPermission((short)0700);
           

          will work. Robert Kanter can we change as above?? any comments ?

          Show
          bibinchundatt Bibin A Chundatt added a comment - RawLocalFileSystem#mkOneDirWithMode applyUMask is failing. In RollingLevelDBTimelineStore static final FsPermission LEVELDB_DIR_UMASK = FsPermission .createImmutable(( short ) 0700); change to static final FsPermission LEVELDB_DIR_UMASK = new FsPermission(( short )0700); will work. Robert Kanter can we change as above?? any comments ?

            People

            • Assignee:
              bibinchundatt Bibin A Chundatt
              Reporter:
              rkanter Robert Kanter
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development