Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.4, 0.95.0
    • Component/s: mapreduce
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Add Bloom Filter support for bulk imports. Lacking a bloom filter, even on a single imported file, can cause perf degradation. Since we now set our compression type based on the HBase CF configuration, it would be good to follow this path for the bloom filter addition.

      1. 3776-0.94.txt
        8 kB
        Lars Hofhansl
      2. 3776-trunk.addendum
        1.0 kB
        Ted Yu
      3. HBASE-3776_v2.patch
        8 kB
        Anoop Sam John
      4. HBASE-3776.patch
        8 kB
        Anoop Sam John

        Issue Links

          Activity

          Hide
          Nicolas Spiegelberg added a comment -

          HBASE-3763 required to easily support HFOF, since we will no longer need to know exact max key count before creating an HFile.

          Show
          Nicolas Spiegelberg added a comment - HBASE-3763 required to easily support HFOF, since we will no longer need to know exact max key count before creating an HFile.
          Hide
          stack added a comment -

          Moving out of 0.92. Move it back in if you think differently.

          Show
          stack added a comment - Moving out of 0.92. Move it back in if you think differently.
          Hide
          stack added a comment -

          Moving out of 0.92. Move it back in if you think differently.

          Show
          stack added a comment - Moving out of 0.92. Move it back in if you think differently.
          Hide
          stack added a comment -

          Making blocker on 0.96 at Nicolas's suggestion

          Show
          stack added a comment - Making blocker on 0.96 at Nicolas's suggestion
          Hide
          stack added a comment -

          Misread. N said 0.94 blocker.

          Show
          stack added a comment - Misread. N said 0.94 blocker.
          Hide
          Lars Hofhansl added a comment -

          Why is this a 0.94 blocker? We lives with this in 0.92.

          Show
          Lars Hofhansl added a comment - Why is this a 0.94 blocker? We lives with this in 0.92.
          Hide
          Lars Hofhansl added a comment -

          Please comment today on why this is a blocker for 0.94. Otherwise I'll move this out of 0.94.

          Show
          Lars Hofhansl added a comment - Please comment today on why this is a blocker for 0.94. Otherwise I'll move this out of 0.94.
          Hide
          Lars Hofhansl added a comment -

          I don't buy that this is a 0.94 blocker. Pull back if you disagree.

          Show
          Lars Hofhansl added a comment - I don't buy that this is a 0.94 blocker. Pull back if you disagree.
          Hide
          Anoop Sam John added a comment -

          @Jerry Any work happening regarding this?

          Show
          Anoop Sam John added a comment - @Jerry Any work happening regarding this?
          Hide
          Chip Salzenberg added a comment -

          Hello? Anyone? Bueller?

          Show
          Chip Salzenberg added a comment - Hello? Anyone? Bueller?
          Hide
          Anoop Sam John added a comment -

          Here the issue was handling of the blooms was happening at the StoreFile level not at the HFile level. The HFileOutputFormat deals with HFiles directly and creates the HFile writers. Can HFileOutputFormat be a StoreFileOutputFormat?
          Just we need to pass the bloom type and the rest will be handled.

          Show
          Anoop Sam John added a comment - Here the issue was handling of the blooms was happening at the StoreFile level not at the HFile level. The HFileOutputFormat deals with HFiles directly and creates the HFile writers. Can HFileOutputFormat be a StoreFileOutputFormat? Just we need to pass the bloom type and the rest will be handled.
          Hide
          Anoop Sam John added a comment -

          @Jerry if you are not working on this , I can take a look at this.

          Show
          Anoop Sam John added a comment - @Jerry if you are not working on this , I can take a look at this.
          Hide
          Jerry Chen added a comment -

          Please. Sorry I wasn't able to get to this.

          Sent from my iPhone

          Show
          Jerry Chen added a comment - Please. Sorry I wasn't able to get to this. Sent from my iPhone
          Hide
          Anoop Sam John added a comment -

          Started working with this. Will give patch next week after test.

          Show
          Anoop Sam John added a comment - Started working with this. Will give patch next week after test.
          Hide
          Anoop Sam John added a comment -

          HFileOutputFormat creates StoreFile writer now so that the handling of the blooms will happen within StoreFile

          Show
          Anoop Sam John added a comment - HFileOutputFormat creates StoreFile writer now so that the handling of the blooms will happen within StoreFile
          Hide
          Ted Yu added a comment -

          Patch looks good.
          TestHFileOutputFormat passes.
          Some minor comments below.

          +  private static Map<byte[], String> createFamilyConfValueMap(Configuration conf, String confName) {
          

          Please add javadoc for the above method.

          +    StringBuilder bloomTypeConfigValue = new StringBuilder();
          +    HTableDescriptor tableDescriptor = table.getTableDescriptor();
          +    if (tableDescriptor == null) {
          +      // could happen with mock table instance
          +      return;
          

          Please move StringBuilder ctor to after the above if block.

          Show
          Ted Yu added a comment - Patch looks good. TestHFileOutputFormat passes. Some minor comments below. + private static Map< byte [], String > createFamilyConfValueMap(Configuration conf, String confName) { Please add javadoc for the above method. + StringBuilder bloomTypeConfigValue = new StringBuilder(); + HTableDescriptor tableDescriptor = table.getTableDescriptor(); + if (tableDescriptor == null ) { + // could happen with mock table instance + return ; Please move StringBuilder ctor to after the above if block.
          Hide
          Anoop Sam John added a comment -

          I will give updated patch today addressing Ted's comments

          Show
          Anoop Sam John added a comment - I will give updated patch today addressing Ted's comments
          Hide
          Anoop Sam John added a comment -

          Submit patch. Let QA run the tests

          Show
          Anoop Sam John added a comment - Submit patch. Let QA run the tests
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561005/HBASE-3776.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any additional warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestMultiParallel

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561005/HBASE-3776.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any additional warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//console This message is automatically generated.
          Hide
          Anoop Sam John added a comment -

          Addressing Ted's comments

          Show
          Anoop Sam John added a comment - Addressing Ted's comments
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561114/HBASE-3776_v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any additional warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestMultiParallel

          -1 core zombie tests. There are zombie tests. See build logs for details.

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561114/HBASE-3776_v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any additional warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel -1 core zombie tests . There are zombie tests. See build logs for details. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Test failure in TestMultiParallel is not related to the patch.

          +1 on v2.

          Show
          Ted Yu added a comment - Test failure in TestMultiParallel is not related to the patch. +1 on v2.
          Hide
          Anoop Sam John added a comment -

          @Ted, @stack - Is it good to go?

          Show
          Anoop Sam John added a comment - @Ted, @stack - Is it good to go?
          Hide
          Ted Yu added a comment -

          I already gave my +1.
          One more +1 is needed.

          Show
          Ted Yu added a comment - I already gave my +1. One more +1 is needed.
          Hide
          Lars Hofhansl added a comment -

          Patch looks good to me +1
          I think we should have this in 0.94 as well.

          Show
          Lars Hofhansl added a comment - Patch looks good to me +1 I think we should have this in 0.94 as well.
          Hide
          Anoop Sam John added a comment -

          Lars
          I can give a patch for 94 version also. Will upload tonight..

          Show
          Anoop Sam John added a comment - Lars I can give a patch for 94 version also. Will upload tonight..
          Hide
          Ted Yu added a comment -

          Integrated to trunk.

          Thanks for the patch, Anoop.

          Thanks for the review, Lars.

          Show
          Ted Yu added a comment - Integrated to trunk. Thanks for the patch, Anoop. Thanks for the review, Lars.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #3664 (See https://builds.apache.org/job/HBase-TRUNK/3664/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #3664 (See https://builds.apache.org/job/HBase-TRUNK/3664/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Ted Yu added a comment -

          Addendum for trunk that fixes reference to BloomType

          Show
          Ted Yu added a comment - Addendum for trunk that fixes reference to BloomType
          Hide
          Lars Hofhansl added a comment -

          Here's a 0.94 patch.

          Show
          Lars Hofhansl added a comment - Here's a 0.94 patch.
          Hide
          Lars Hofhansl added a comment -

          Opps @Anop. Just saw you latest comment now. Anyway, please review the 0.94 patch I attached.

          Show
          Lars Hofhansl added a comment - Opps @Anop. Just saw you latest comment now. Anyway, please review the 0.94 patch I attached.
          Hide
          Anoop Sam John added a comment -

          Thanks for that addendum Ted.
          Lars thanks for the port. +1

          Show
          Anoop Sam John added a comment - Thanks for that addendum Ted. Lars thanks for the port. +1
          Hide
          Lars Hofhansl added a comment -

          I ran TestHFileOutputFormat in 0.94.
          Patch's almost identical to trunks, committing soon, unless anybody objects.

          Show
          Lars Hofhansl added a comment - I ran TestHFileOutputFormat in 0.94. Patch's almost identical to trunks, committing soon, unless anybody objects.
          Hide
          Lars Hofhansl added a comment -

          Committed to 0.94 as well. Thanks for the patch Anoop.

          Show
          Lars Hofhansl added a comment - Committed to 0.94 as well. Thanks for the patch Anoop.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #3665 (See https://builds.apache.org/job/HBase-TRUNK/3665/)
          HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #3665 (See https://builds.apache.org/job/HBase-TRUNK/3665/ ) HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #673 (See https://builds.apache.org/job/HBase-0.94/673/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #673 (See https://builds.apache.org/job/HBase-0.94/673/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #316 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/316/)
          HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java

          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #316 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/316/ ) HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Ted Yu added a comment -

          looks like this Jira is related to the javac warning issue in recent PreCommit builds.

          Show
          Ted Yu added a comment - looks like this Jira is related to the javac warning issue in recent PreCommit builds.
          Hide
          Ted Yu added a comment -

          The above comment was inaccurate: javac warning issue was due mvn command not found on hadoop9.

          Show
          Ted Yu added a comment - The above comment was inaccurate: javac warning issue was due mvn command not found on hadoop9.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #90 (See https://builds.apache.org/job/HBase-0.94-security/90/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #90 (See https://builds.apache.org/job/HBase-0.94-security/90/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java

            People

            • Assignee:
              Anoop Sam John
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development