HBase
  1. HBase
  2. HBASE-5213

"hbase master stop" does not bring down backup masters

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.90.5, 0.92.0, 0.94.0, 0.95.2
    • Fix Version/s: 0.94.0, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Typing "hbase master stop" produces the following message:

      "stop Start cluster shutdown; Master signals RegionServer shutdown"
      It seems like backup masters should be considered part of the cluster, but they are not brought down by "hbase master stop".
      "stop-hbase.sh" does correctly bring down the backup masters.

      The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() – this isn't too surprising since I think "hbase master stop" just calls this API.

      It seems like HBASE-1448 address this; perhaps there was a regression?

      1. HBASE-5213-v2-trunk.patch
        13 kB
        Gregory Chanan
      2. HBASE-5213-v2-92.patch
        13 kB
        Gregory Chanan
      3. HBASE-5213-v2-90.patch
        13 kB
        Gregory Chanan
      4. HBASE-5213-v1-trunk.patch
        13 kB
        Gregory Chanan
      5. HBASE-5213-v0-trunk.patch
        13 kB
        Gregory Chanan
      6. 5213.jstack
        135 kB
        Ted Yu

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          Ted Yu added a comment -

          In stop-hbase.sh, I see:

            # TODO: store backup masters in ZooKeeper and have the primary send them a shutdown message
            # stop any backup masters
            "$bin"/hbase-daemons.sh --config "${HBASE_CONF_DIR}" \
              --hosts "${HBASE_BACKUP_MASTERS}" stop master-backup
          

          We have several options:
          1. adding a "shutdown node" in ZK on which all masters should listen
          2. introducing "backup masters" node in ZK so that we know where shutdown message should be sent

          Show
          Ted Yu added a comment - In stop-hbase.sh, I see: # TODO: store backup masters in ZooKeeper and have the primary send them a shutdown message # stop any backup masters "$bin" /hbase-daemons.sh --config "${HBASE_CONF_DIR}" \ --hosts "${HBASE_BACKUP_MASTERS}" stop master-backup We have several options: 1. adding a "shutdown node" in ZK on which all masters should listen 2. introducing "backup masters" node in ZK so that we know where shutdown message should be sent
          Hide
          Gregory Chanan added a comment -

          This fixes the issue and includes a new unit test. I also tested on a cluster using "hbase master stop" and the backup masters properly shutdown.

          If this is acceptable I'll provide patches for previous versions.

          Show
          Gregory Chanan added a comment - Attached HBASE-5213 -v0-trunk.patch * This fixes the issue and includes a new unit test. I also tested on a cluster using "hbase master stop" and the backup masters properly shutdown. If this is acceptable I'll provide patches for previous versions.
          Hide
          Ted Yu added a comment -

          The approach is good.
          Formatting needs to improve in several places. Such as:

          +      if(!clusterStatusTracker.isClusterUp()) {
          +     this.master.stop("Cluster went down before this master became active");
          +      }
          

          A space should be added between if and (
          Indentation is a little off.

          + * Copyright 2012 The Apache Software Foundation
          

          No year is needed above.

          Show
          Ted Yu added a comment - The approach is good. Formatting needs to improve in several places. Such as: + if (!clusterStatusTracker.isClusterUp()) { + this .master.stop( "Cluster went down before this master became active" ); + } A space should be added between if and ( Indentation is a little off. + * Copyright 2012 The Apache Software Foundation No year is needed above.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12517498/HBASE-5213-v0-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -129 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.mapreduce.TestImportTsv
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
          org.apache.hadoop.hbase.master.TestSplitLogManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517498/HBASE-5213-v0-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -129 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1130//console This message is automatically generated.
          Hide
          Gregory Chanan added a comment -

          Address formatting issues with prior patch.

          Show
          Gregory Chanan added a comment - Attached HBASE-5213 -v1-trunk.patch * Address formatting issues with prior patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12517524/HBASE-5213-v1-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -129 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
          org.apache.hadoop.hbase.client.TestAdmin
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestImportTsv

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517524/HBASE-5213-v1-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -129 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1132//console This message is automatically generated.
          Hide
          Gregory Chanan added a comment -
          • Attached v2 for trunk,92, and 90 *

          For trunk, only small differences from previous version (comments and spacing).

          Tested that trunk patch works with 94 as well.

          Created patches for 92 and 90.

          Show
          Gregory Chanan added a comment - Attached v2 for trunk,92, and 90 * For trunk, only small differences from previous version (comments and spacing). Tested that trunk patch works with 94 as well. Created patches for 92 and 90.
          Hide
          Ted Yu added a comment -

          I ran TestAdmin three times based on patch v2.
          They passed.

          Show
          Ted Yu added a comment - I ran TestAdmin three times based on patch v2. They passed.
          Hide
          Ted Yu added a comment -

          Will integrated TRUNK patch to 0.94 and TRUNK if there is no objection.

          Show
          Ted Yu added a comment - Will integrated TRUNK patch to 0.94 and TRUNK if there is no objection.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12517644/HBASE-5213-v2-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -123 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.coprocessor.TestMasterObserver
          org.apache.hadoop.hbase.mapreduce.TestImportTsv
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517644/HBASE-5213-v2-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestMasterObserver org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1140//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Integrated to TRUNK.

          Thanks for the patch Gregory.

          Show
          Ted Yu added a comment - Integrated to TRUNK. Thanks for the patch Gregory.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2675 (See https://builds.apache.org/job/HBase-TRUNK/2675/)
          HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298859)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2675 (See https://builds.apache.org/job/HBase-TRUNK/2675/ ) HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298859) Result = SUCCESS tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Hide
          Ted Yu added a comment -

          Integrated to 0.94 as well.

          Show
          Ted Yu added a comment - Integrated to 0.94 as well.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #23 (See https://builds.apache.org/job/HBase-0.94/23/)
          HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298877)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #23 (See https://builds.apache.org/job/HBase-0.94/23/ ) HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298877) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Hide
          Gregory Chanan added a comment -

          Any thoughts regarding 92 or 90?

          Thanks for the reviews/commit.

          Show
          Gregory Chanan added a comment - Any thoughts regarding 92 or 90? Thanks for the reviews/commit.
          Hide
          stack added a comment -

          Marking as for 0.92.2 (of if a new 0.92.1 RC, will include it in that)

          Show
          stack added a comment - Marking as for 0.92.2 (of if a new 0.92.1 RC, will include it in that)
          Hide
          Gregory Chanan added a comment -

          @stack:

          You wrote 0.92.2 but marked as 0.90.2. Changed to 0.92.2

          Show
          Gregory Chanan added a comment - @stack: You wrote 0.92.2 but marked as 0.90.2. Changed to 0.92.2
          Hide
          stack added a comment -

          Thanks G.

          Show
          stack added a comment - Thanks G.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #133 (See https://builds.apache.org/job/HBase-TRUNK-security/133/)
          HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298859)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #133 (See https://builds.apache.org/job/HBase-TRUNK-security/133/ ) HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1298859) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Hide
          Jonathan Hsieh added a comment -

          Noticed that this was committed to 0.94 and 0.96/trunk but not to 0.92 or 0.90. running some tests on thos verions and if clean will commit.

          Show
          Jonathan Hsieh added a comment - Noticed that this was committed to 0.94 and 0.96/trunk but not to 0.92 or 0.90. running some tests on thos verions and if clean will commit.
          Hide
          Jonathan Hsieh added a comment -

          Committed to 0.92 and 0.90. Thanks Greg.

          Show
          Jonathan Hsieh added a comment - Committed to 0.92 and 0.90. Thanks Greg.
          Hide
          Jonathan Hsieh added a comment -

          Committed to 0.92 and 0.90. Thanks Greg.

          Show
          Jonathan Hsieh added a comment - Committed to 0.92 and 0.90. Thanks Greg.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #348 (See https://builds.apache.org/job/HBase-0.92/348/)
          HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1308012)

          Result = SUCCESS
          jmhsieh :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Show
          Hudson added a comment - Integrated in HBase-0.92 #348 (See https://builds.apache.org/job/HBase-0.92/348/ ) HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1308012) Result = SUCCESS jmhsieh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92-security #104 (See https://builds.apache.org/job/HBase-0.92-security/104/)
          HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1308012)

          Result = FAILURE
          jmhsieh :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Show
          Hudson added a comment - Integrated in HBase-0.92-security #104 (See https://builds.apache.org/job/HBase-0.92-security/104/ ) HBASE-5213 "hbase master stop" does not bring down backup masters (Gregory) (Revision 1308012) Result = FAILURE jmhsieh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
          Hide
          stack added a comment -

          This seems to have broken 0.90 builds. Can we revert from 0.90 trunk till fixed. See here: https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/ Build #458 is where it got committed and thereafter the build fails rolling the wal test.

          Show
          stack added a comment - This seems to have broken 0.90 builds. Can we revert from 0.90 trunk till fixed. See here: https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/ Build #458 is where it got committed and thereafter the build fails rolling the wal test.
          Hide
          David S. Wang added a comment -

          Greg is out of town for the next week or so ... IMHO if it's breaking 0.90, we should revert for now and Greg can look at it when he gets back.

          Show
          David S. Wang added a comment - Greg is out of town for the next week or so ... IMHO if it's breaking 0.90, we should revert for now and Greg can look at it when he gets back.
          Hide
          Ted Yu added a comment -

          TestLogRolling hangs in 0.90

          Show
          Ted Yu added a comment - TestLogRolling hangs in 0.90
          Hide
          Ted Yu added a comment -

          jstack for the hanging test.

          Show
          Ted Yu added a comment - jstack for the hanging test.
          Hide
          Jonathan Hsieh added a comment -

          I'll revert the 0.90 version. Sorry about this fellas.

          Show
          Jonathan Hsieh added a comment - I'll revert the 0.90 version. Sorry about this fellas.
          Hide
          Lars Hofhansl added a comment -

          So from the discussion the test is only a problem in 0.90? Do we know why this is?
          Asking because we're close to another RC attempt for 0.94.0.

          Show
          Lars Hofhansl added a comment - So from the discussion the test is only a problem in 0.90? Do we know why this is? Asking because we're close to another RC attempt for 0.94.0.
          Hide
          Jonathan Hsieh added a comment -

          The issue is apparently only in 0.90. How about we close this issue for 0.92/0.94/trunk and create a follow on issue for 0.90? This will unblock this for 0.94 and Greg can address this when he gets back.

          Show
          Jonathan Hsieh added a comment - The issue is apparently only in 0.90. How about we close this issue for 0.92/0.94/trunk and create a follow on issue for 0.90? This will unblock this for 0.94 and Greg can address this when he gets back.
          Hide
          Lars Hofhansl added a comment -

          Marking this one as fixed. Created sub-issue for 0.90 discussion.

          Show
          Lars Hofhansl added a comment - Marking this one as fixed. Created sub-issue for 0.90 discussion.

            People

            • Assignee:
              Gregory Chanan
              Reporter:
              Gregory Chanan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development