HBase
  1. HBase
  2. HBASE-5711

Tests are failing with incorrect data directory permissions.

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.92.3
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When we run some tests in Hbase (TestAdmin), it is failing with following error.

      Starting DataNode 0 with dfs.data.dir: E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data1,E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data2
      2012-04-04 18:04:51,036 WARN [main] impl.MetricsSystemImpl(137): Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
      2012-04-04 18:04:51,255 WARN [main] datanode.DataNode(1548): Invalid directory in dfs.data.dir: Incorrect permission for E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data1, expected: rwxr-xr-x, while actual: rwx------
      2012-04-04 18:04:51,411 WARN [main] datanode.DataNode(1548): Invalid directory in dfs.data.dir: Incorrect permission for E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data2, expected: rwxr-xr-x, while actual: rwx------
      2012-04-04 18:04:51,411 ERROR [main] datanode.DataNode(1554): All directories in dfs.data.dir are invalid.
      2012-04-04 18:04:51,411 INFO [main] hbase.HBaseTestingUtility(684): Shutting down minicluster
      2012-04-04 18:04:51,646 WARN [main] hbase.HBaseTestingUtility(696): Failed delete of E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb
      2012-04-04 18:04:51,646 INFO [main] hbase.HBaseTestingUtility(700): Minicluster is down

      1. HBASE-5711.patch
        0.7 kB
        Uma Maheswara Rao G

        Activity

        Hide
        Uma Maheswara Rao G added a comment -

        looks like we have to set "dfs.datanode.data.dir.perm" to 700.

        Tests are running properly after setting this property in cluster conf.

        Show
        Uma Maheswara Rao G added a comment - looks like we have to set "dfs.datanode.data.dir.perm" to 700. Tests are running properly after setting this property in cluster conf.
        Hide
        Uma Maheswara Rao G added a comment -

        Attached simple patch, which sets the permission to dfs data directories with 700.

        Show
        Uma Maheswara Rao G added a comment - Attached simple patch, which sets the permission to dfs data directories with 700.
        Hide
        Uma Maheswara Rao G added a comment -

        @Ram, could you please take a look?

        Show
        Uma Maheswara Rao G added a comment - @Ram, could you please take a look?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521350/HBASE-5711.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521350/HBASE-5711.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1388//console This message is automatically generated.
        Hide
        ramkrishna.s.vasudevan added a comment -

        +1 on patch. This patch is needed so that test cases can be run locally.
        Thanks Uma for the patch.

        Show
        ramkrishna.s.vasudevan added a comment - +1 on patch. This patch is needed so that test cases can be run locally. Thanks Uma for the patch.
        Hide
        Lewis John McGibbney added a comment -

        We do not see the ERROR but certainly see the WARN's when using HBaseTestingUtility across MR in our Apache Gora tests [0]. We see many

        12/04/05 16:56:05 WARN impl.MetricsSystemImpl: Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-namenode.properties, hadoop-metrics2.properties
        

        [0] https://svn.apache.org/viewvc/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java?view=markup

        Show
        Lewis John McGibbney added a comment - We do not see the ERROR but certainly see the WARN's when using HBaseTestingUtility across MR in our Apache Gora tests [0] . We see many 12/04/05 16:56:05 WARN impl.MetricsSystemImpl: Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-namenode.properties, hadoop-metrics2.properties [0] https://svn.apache.org/viewvc/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java?view=markup
        Hide
        Uma Maheswara Rao G added a comment -

        @Ram, Thanks a lot for review!

        @Lewis, this is filed for permissions related problem. Unfortunately trace contains this warning as well. Not for metrics system warning. I think, if you are not looking for any metrics, you need not worry about that warning.

        Show
        Uma Maheswara Rao G added a comment - @Ram, Thanks a lot for review! @Lewis, this is filed for permissions related problem. Unfortunately trace contains this warning as well. Not for metrics system warning. I think, if you are not looking for any metrics, you need not worry about that warning.
        Hide
        Lewis John McGibbney added a comment -

        My apologies, I found it elsewhere HBASE-4709

        Show
        Lewis John McGibbney added a comment - My apologies, I found it elsewhere HBASE-4709
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #158 (See https://builds.apache.org/job/HBase-TRUNK-security/158/)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310050)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #158 (See https://builds.apache.org/job/HBase-TRUNK-security/158/ ) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310050) Result = FAILURE stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2715 (See https://builds.apache.org/job/HBase-TRUNK/2715/)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310050)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2715 (See https://builds.apache.org/job/HBase-TRUNK/2715/ ) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310050) Result = FAILURE stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #356 (See https://builds.apache.org/job/HBase-0.92/356/)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310053)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #356 (See https://builds.apache.org/job/HBase-0.92/356/ ) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310053) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #90 (See https://builds.apache.org/job/HBase-0.94/90/)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310051)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #90 (See https://builds.apache.org/job/HBase-0.94/90/ ) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310051) Result = FAILURE stack : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Ted Yu added a comment -

        @ 04/Apr/12 18:30, TestRowProcessorEndpoint was reported by Hadoop QA to fail.

        After I reverted the patch, TestRowProcessorEndpoint passed on trunk.

        Show
        Ted Yu added a comment - @ 04/Apr/12 18:30, TestRowProcessorEndpoint was reported by Hadoop QA to fail. After I reverted the patch, TestRowProcessorEndpoint passed on trunk.
        Hide
        stack added a comment -

        Yeah, just noticed this myself trying another fix. Reverted from trunk and 0.94.

        Show
        stack added a comment - Yeah, just noticed this myself trying another fix. Reverted from trunk and 0.94.
        Hide
        Ted Yu added a comment -

        Thanks for the quick response.

        What about 0.92 ?

        Show
        Ted Yu added a comment - Thanks for the quick response. What about 0.92 ?
        Hide
        stack added a comment -

        Done. Thanks for reminder.

        Show
        stack added a comment - Done. Thanks for reminder.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2716 (See https://builds.apache.org/job/HBase-TRUNK/2716/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310102)
        HBASE-5711 Tests are failing with incorrect data directory permissions – OVER-REVERTED; UNDOING LAST COMMIT (Revision 1310097)
        HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310095)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java

        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java

        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2716 (See https://builds.apache.org/job/HBase-TRUNK/2716/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310102) HBASE-5711 Tests are failing with incorrect data directory permissions – OVER-REVERTED; UNDOING LAST COMMIT (Revision 1310097) HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310095) Result = SUCCESS stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #91 (See https://builds.apache.org/job/HBase-0.94/91/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310101)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #91 (See https://builds.apache.org/job/HBase-0.94/91/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310101) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #357 (See https://builds.apache.org/job/HBase-0.92/357/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310107)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #357 (See https://builds.apache.org/job/HBase-0.92/357/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310107) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Jonathan Hsieh added a comment -

        I've been working around this by setting 'umask 022' in my shell before executing tests/eclipse, etc.

        Show
        Jonathan Hsieh added a comment - I've been working around this by setting 'umask 022' in my shell before executing tests/eclipse, etc.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #159 (See https://builds.apache.org/job/HBase-TRUNK-security/159/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310102)
        HBASE-5711 Tests are failing with incorrect data directory permissions – OVER-REVERTED; UNDOING LAST COMMIT (Revision 1310097)
        HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310095)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java

        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java

        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #159 (See https://builds.apache.org/job/HBase-TRUNK-security/159/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310102) HBASE-5711 Tests are failing with incorrect data directory permissions – OVER-REVERTED; UNDOING LAST COMMIT (Revision 1310097) HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310095) Result = FAILURE stack : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
        Hide
        ramkrishna.s.vasudevan added a comment -

        Surprising that the tests are failing. Without the patch we get nullpointer exception and the stack trace saying invalid directories.

        With patch we are able to run the tests cleanly.

        Show
        ramkrishna.s.vasudevan added a comment - Surprising that the tests are failing. Without the patch we get nullpointer exception and the stack trace saying invalid directories. With patch we are able to run the tests cleanly.
        Hide
        Uma Maheswara Rao G added a comment -

        Strange....I ran several times this tests, passes for me. Still tests are failing without this patch. Let me check again, if I get some clue.

        @Jon

        I've been working around this by setting 'umask 022' in my shell before executing tests/eclipse, etc.

        We are woking on windows/eclipse for dev env. Where exactly you are modifying this to take effect in eclipse execution? I guess your dev env might be linux?

        Show
        Uma Maheswara Rao G added a comment - Strange....I ran several times this tests, passes for me. Still tests are failing without this patch. Let me check again, if I get some clue. @Jon I've been working around this by setting 'umask 022' in my shell before executing tests/eclipse, etc. We are woking on windows/eclipse for dev env. Where exactly you are modifying this to take effect in eclipse execution? I guess your dev env might be linux?
        Hide
        Jonathan Hsieh added a comment -

        @Uma
        Yeah, I'm a linux+eclipse guy for dev environment – it is just in scripts for running tests or in my login script before I kick off eclipse. I think I answered question on the mailing list about this the person came back saying they were in windows as well.

        Show
        Jonathan Hsieh added a comment - @Uma Yeah, I'm a linux+eclipse guy for dev environment – it is just in scripts for running tests or in my login script before I kick off eclipse. I think I answered question on the mailing list about this the person came back saying they were in windows as well.
        Hide
        ramkrishna.s.vasudevan added a comment -

        @Jon
        It was me who sent that mail in the mailing list.

        Show
        ramkrishna.s.vasudevan added a comment - @Jon It was me who sent that mail in the mailing list.
        Hide
        Lars Hofhansl added a comment -

        We had this problem forever. I posted a message on this to the mailing on October 27th.
        This is not in the way of an RC1 for 0.94.0.

        Show
        Lars Hofhansl added a comment - We had this problem forever. I posted a message on this to the mailing on October 27th. This is not in the way of an RC1 for 0.94.0.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310101)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310051)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java

        stack :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – RE-REVERT (Revision 1310101) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310051) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java stack : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/)
        HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310107)
        HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310053)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java

        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Show
        Hudson added a comment - Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/ ) HBASE-5711 Tests are failing with incorrect data directory permissions – REVERT (Revision 1310107) HBASE-5711 Tests are failing with incorrect data directory permissions. (Revision 1310053) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        Hide
        Dave Revell added a comment -

        Here's a workaround for people running into permission problems while embedding a minicluster.

                    hbaseTestUtil = new HBaseTestingUtility();
        
                    // Workaround for HBASE-5711, we need to set config value dfs.datanode.data.dir.perm
                    // equal to the permissions of the temp dirs on the filesystem. These temp dirs were
                    // probably created using this process' umask. So we guess the temp dir permissions as
                    // 0777 & ~umask, and use that to set the config value.
                    try {
                        Process process = Runtime.getRuntime().exec("/bin/sh -c umask");
                        BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()));
                        int rc = process.waitFor();
                        if(rc == 0) {
                            String umask = br.readLine();
            
                            int umaskBits = Integer.parseInt(umask, 8);
                            int permBits = 0777 & ~umaskBits;
                            String perms = Integer.toString(permBits, 8);
                            
                            log.info("Setting dfs.datanode.data.dir.perm to " + perms);
                            hbaseTestUtil.getConfiguration().set("dfs.datanode.data.dir.perm", perms);
                        } else {
                            log.warn("Failed running umask command in a shell, nonzero return value");
                        }
                    } catch (Exception e) {
                        // ignore errors, we might not be running on POSIX, or "sh" might not be on the path
                        log.warn("Couldn't get umask", e);
                    }
                    
                    hbaseTestUtil.startMiniCluster();
        
        Show
        Dave Revell added a comment - Here's a workaround for people running into permission problems while embedding a minicluster. hbaseTestUtil = new HBaseTestingUtility(); // Workaround for HBASE-5711, we need to set config value dfs.datanode.data.dir.perm // equal to the permissions of the temp dirs on the filesystem. These temp dirs were // probably created using this process' umask. So we guess the temp dir permissions as // 0777 & ~umask, and use that to set the config value. try { Process process = Runtime.getRuntime().exec("/bin/sh -c umask"); BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream())); int rc = process.waitFor(); if(rc == 0) { String umask = br.readLine(); int umaskBits = Integer.parseInt(umask, 8); int permBits = 0777 & ~umaskBits; String perms = Integer.toString(permBits, 8); log.info("Setting dfs.datanode.data.dir.perm to " + perms); hbaseTestUtil.getConfiguration().set("dfs.datanode.data.dir.perm", perms); } else { log.warn("Failed running umask command in a shell, nonzero return value"); } } catch (Exception e) { // ignore errors, we might not be running on POSIX, or "sh" might not be on the path log.warn("Couldn't get umask", e); } hbaseTestUtil.startMiniCluster();

          People

          • Assignee:
            Uma Maheswara Rao G
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development