ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1878

Inconsistent behavior in autocreation of dataDir and dataLogDir

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.4.5
    • Fix Version/s: 3.4.7, 3.5.0
    • Component/s: quorum
    • Labels:
      None

      Description

      During the startup if dataDir is not exists server will auto create this. But when user specifies different dataLogDir location which doesn't exists the server will validate and startup will fail.

      org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing build\test3085582797504170966.junit.dir\zoo.cfg
      	at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:123)
      	at org.apache.zookeeper.server.ServerConfig.parse(ServerConfig.java:79)
      	at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:81)
      	at org.apache.zookeeper.server.ZooKeeperServerMainTest$MainThread.run(ZooKeeperServerMainTest.java:92)
      Caused by: java.lang.IllegalArgumentException: dataLogDir build/test3085582797504170966.junit.dir/data_txnlog is missing.
      	at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:253)
      	at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:119)
      	... 3 more
      
      1. ZOOKEEPER-1878_br_3_4.patch
        4 kB
        Rakesh R
      2. ZOOKEEPER-1878_br_3_4.patch
        4 kB
        Rakesh R
      3. ZOOKEEPER-1878.patch
        5 kB
        Rakesh R
      4. ZOOKEEPER-1878.patch
        5 kB
        Rakesh R
      5. ZOOKEEPER-1878-resets-autocreate-flag.patch
        2 kB
        Rakesh R

        Activity

        Rakesh R created issue -
        Rakesh R made changes -
        Field Original Value New Value
        Attachment ZOOKEEPER-1878.patch [ 12627845 ]
        Rakesh R made changes -
        Attachment ZOOKEEPER-1878.patch [ 12627845 ]
        Rakesh R made changes -
        Attachment ZOOKEEPER-1878_br_3_4.patch [ 12627846 ]
        Rakesh R made changes -
        Attachment ZOOKEEPER-1878.patch [ 12627847 ]
        Rakesh R made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 3.4.5 [ 12321883 ]
        Fix Version/s 3.4.6 [ 12323310 ]
        Fix Version/s 3.5.0 [ 12316644 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12627847/ZOOKEEPER-1878.patch
        against trunk revision 1561672.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12627847/ZOOKEEPER-1878.patch against trunk revision 1561672. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1922//console This message is automatically generated.
        Hide
        Rakesh R added a comment -
        -1 core tests. The patch failed core unit tests.
        

        Following is the test case failure reported by QA. There is an issue ZOOKEEPER-1870 open to address the same, so kindly ignore this failure report.

        junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds.
        	at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:143)
        	at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest(StandaloneDisabledTest.java:75)
        	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
        

        Please review the problem and the proposed patch. Thanks

        Show
        Rakesh R added a comment - -1 core tests. The patch failed core unit tests. Following is the test case failure reported by QA. There is an issue ZOOKEEPER-1870 open to address the same, so kindly ignore this failure report. junit.framework.AssertionFailedError: client could not connect to reestablished quorum: giving up after 30+ seconds. at org.apache.zookeeper.test.ReconfigTest.testNormalOperation(ReconfigTest.java:143) at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest(StandaloneDisabledTest.java:75) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52) Please review the problem and the proposed patch. Thanks
        Flavio Junqueira made changes -
        Fix Version/s 3.4.7 [ 12325149 ]
        Fix Version/s 3.4.6 [ 12323310 ]
        Hide
        Michi Mutsuzaki added a comment -

        Hi Rakesh,

        Could you put the patch on the reviewboard? I have some questions/comments.

        Thanks!
        --Michi

        Show
        Michi Mutsuzaki added a comment - Hi Rakesh, Could you put the patch on the reviewboard? I have some questions/comments. Thanks! --Michi
        Hide
        Rakesh R added a comment -

        Thanks Michi Mutsuzaki for the interest. Please see the review https://reviews.apache.org/r/19089/

        Show
        Rakesh R added a comment - Thanks Michi Mutsuzaki for the interest. Please see the review https://reviews.apache.org/r/19089/
        Rakesh R made changes -
        Attachment ZOOKEEPER-1878.patch [ 12634722 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12634722/ZOOKEEPER-1878.patch
        against trunk revision 1577317.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634722/ZOOKEEPER-1878.patch against trunk revision 1577317. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1962//console This message is automatically generated.
        Hide
        Rakesh R added a comment -

        Attached new patch by fixing the comments given in the review board. Thanks again Raul Gutierrez Segales, Camille Fournier, Michi Mutsuzaki for the reviews.

        Show
        Rakesh R added a comment - Attached new patch by fixing the comments given in the review board. Thanks again Raul Gutierrez Segales , Camille Fournier , Michi Mutsuzaki for the reviews.
        Hide
        Raul Gutierrez Segales added a comment -

        lgtm, +1 - thanks Rakesh.

        Show
        Raul Gutierrez Segales added a comment - lgtm, +1 - thanks Rakesh.
        Hide
        Michi Mutsuzaki added a comment -

        trunk: http://svn.apache.org/viewvc?view=revision&revision=1577750

        Rakesh, could you update the patch for the 3.4 branch?

        Show
        Michi Mutsuzaki added a comment - trunk: http://svn.apache.org/viewvc?view=revision&revision=1577750 Rakesh, could you update the patch for the 3.4 branch?
        Rakesh R made changes -
        Attachment ZOOKEEPER-1878_br_3_4.patch [ 12634888 ]
        Hide
        Rakesh R added a comment -

        Michi Mutsuzaki Attached 3.4 patch, please have a look. Thanks

        Show
        Rakesh R added a comment - Michi Mutsuzaki Attached 3.4 patch, please have a look. Thanks
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12634888/ZOOKEEPER-1878_br_3_4.patch
        against trunk revision 1577756.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1963//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634888/ZOOKEEPER-1878_br_3_4.patch against trunk revision 1577756. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1963//console This message is automatically generated.
        Hide
        Michi Mutsuzaki added a comment -
        Show
        Michi Mutsuzaki added a comment - branch 3.4: http://svn.apache.org/viewvc?view=revision&revision=1577789 Thanks Rakesh!
        Michi Mutsuzaki made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Rakesh R added a comment -

        Re-opening the issue to address the following in tests :

        At the beginning of ZooKeeperServerMainTest#testWithoutAutoCreateDataLogDir, its setting system variable to false and not resetting the same at the end of the testcase exec. This would affect other test cases and hits “Missing data directory” Exception, if they are executing after this testcase.

        Show
        Rakesh R added a comment - Re-opening the issue to address the following in tests : At the beginning of ZooKeeperServerMainTest#testWithoutAutoCreateDataLogDir, its setting system variable to false and not resetting the same at the end of the testcase exec. This would affect other test cases and hits “Missing data directory” Exception, if they are executing after this testcase.
        Rakesh R made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Rakesh R made changes -
        Hide
        Rakesh R added a comment -

        Hi Michi Mutsuzaki, As per our discussion(offline) I had re-opened the issue. Kindly see the attached patch, where I'm resetting the flag to default value, so that others will not get affected. Thanks.

        Show
        Rakesh R added a comment - Hi Michi Mutsuzaki , As per our discussion(offline) I had re-opened the issue. Kindly see the attached patch, where I'm resetting the flag to default value, so that others will not get affected. Thanks.
        Rakesh R made changes -
        Status Reopened [ 4 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12636969/ZOOKEEPER-1878-resets-autocreate-flag.patch
        against trunk revision 1581522.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636969/ZOOKEEPER-1878-resets-autocreate-flag.patch against trunk revision 1581522. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1986//console This message is automatically generated.
        Hide
        Michi Mutsuzaki added a comment -

        trunk: http://svn.apache.org/viewvc?view=revision&revision=1582067

        Rakesh, does this also affect 3.4 branch?

        Show
        Michi Mutsuzaki added a comment - trunk: http://svn.apache.org/viewvc?view=revision&revision=1582067 Rakesh, does this also affect 3.4 branch?
        Hide
        Rakesh R added a comment -

        Thanks Michi Mutsuzaki. "zookeeper.datadir.autocreate" flag is not present in 3.4 and no changes required in 3.4 branch.

        Show
        Rakesh R added a comment - Thanks Michi Mutsuzaki . "zookeeper.datadir.autocreate" flag is not present in 3.4 and no changes required in 3.4 branch.
        Hide
        Michi Mutsuzaki added a comment -

        Ok thanks Rakesh!

        Show
        Michi Mutsuzaki added a comment - Ok thanks Rakesh!
        Michi Mutsuzaki made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in ZooKeeper-trunk #2267 (See https://builds.apache.org/job/ZooKeeper-trunk/2267/)
        ZOOKEEPER-1878. Reset zookeeper.datadir.autocreate (Rakesh R via michim) (michim: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582067)

        • /zookeeper/trunk/src/java/test/org/apache/zookeeper/server/ZooKeeperServerMainTest.java
        Show
        Hudson added a comment - SUCCESS: Integrated in ZooKeeper-trunk #2267 (See https://builds.apache.org/job/ZooKeeper-trunk/2267/ ) ZOOKEEPER-1878 . Reset zookeeper.datadir.autocreate (Rakesh R via michim) (michim: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582067 ) /zookeeper/trunk/src/java/test/org/apache/zookeeper/server/ZooKeeperServerMainTest.java

          People

          • Assignee:
            Rakesh R
            Reporter:
            Rakesh R
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development