Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18994

Decide if META/System tables should use Compacting Memstore or Default Memstore

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha-3
    • Fix Version/s: 2.0.0-alpha-4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added a new config 'hbase.systemtables.compacting.memstore.type" for the system tables. By default all the system tables will have 'NONE' as the type and so it will be using the default memstore by default.
      {code}
       <property>
          <name>hbase.systemtables.compacting.memstore.type</name>
          <value>NONE</value>
        </property>
      {code}
      Show
      Added a new config 'hbase.systemtables.compacting.memstore.type" for the system tables. By default all the system tables will have 'NONE' as the type and so it will be using the default memstore by default. {code}  <property>     <name>hbase.systemtables.compacting.memstore.type</name>     <value>NONE</value>   </property> {code}

      Description

      HBASE-18992 brought this topic. Currently META is also using Compacting Memstore. We should decide if system tables can go with Default memstore only.

      1. HBASE-18994_3.patch
        3 kB
        ramkrishna.s.vasudevan
      2. HBASE-18994_2.patch
        3 kB
        ramkrishna.s.vasudevan
      3. HBASE-18994_1.patch
        3 kB
        ramkrishna.s.vasudevan
      4. HBASE-18994.patch
        3 kB
        ramkrishna.s.vasudevan

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build HBase-2.0 #750 (See https://builds.apache.org/job/HBase-2.0/750/)
          HBASE-18994 Decide if META/System tables should use Compacting Memstore (ramkrishna: rev 015db0a7aa97398ceae79bd9d5fd34aec1bdc463)

          • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-2.0 #750 (See https://builds.apache.org/job/HBase-2.0/750/ ) HBASE-18994 Decide if META/System tables should use Compacting Memstore (ramkrishna: rev 015db0a7aa97398ceae79bd9d5fd34aec1bdc463) (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3950 (See https://builds.apache.org/job/HBase-Trunk_matrix/3950/)
          HBASE-18994 Decide if META/System tables should use Compacting Memstore (ramkrishna: rev 0acfba0e35933d1008c9ade721055dafc137b50f)

          • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3950 (See https://builds.apache.org/job/HBase-Trunk_matrix/3950/ ) HBASE-18994 Decide if META/System tables should use Compacting Memstore (ramkrishna: rev 0acfba0e35933d1008c9ade721055dafc137b50f) (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Oh . Just seeing your comment. Wil push the the hbase-default.xml change as part of sub-JIRA.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Oh . Just seeing your comment. Wil push the the hbase-default.xml change as part of sub-JIRA.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Pushed to master and branch-2.
          Thanks for the reivews. Will raise a subJIRA to add to docs/hbase-default.xml. Yes we can start a dev@ mail to discuss on the creation of tables in system namespace.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Pushed to master and branch-2. Thanks for the reivews. Will raise a subJIRA to add to docs/hbase-default.xml. Yes we can start a dev@ mail to discuss on the creation of tables in system namespace.
          Hide
          stack stack added a comment - - edited

          +1 on commit. Add to hbase-default.xml on commit. Add a nice description. This'll make it show up in docs (we import hbase-default.xml when we generate the refguide).

          Show
          stack stack added a comment - - edited +1 on commit. Add to hbase-default.xml on commit. Add a nice description. This'll make it show up in docs (we import hbase-default.xml when we generate the refguide).
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          So for now I will add a release note and commit this? So we need a doc update I believe and is there a need to add to hbase-default.xml? Let me know I shall commit this then.
          Thanks all for the reviews.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - So for now I will add a release note and commit this? So we need a doc update I believe and is there a need to add to hbase-default.xml? Let me know I shall commit this then. Thanks all for the reviews.
          Hide
          stack stack added a comment -

          +1 on patch.. Make note of new config here in release note.

          Show
          stack stack added a comment - +1 on patch.. Make note of new config here in release note.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          +1 for dev@ discussion.. Ya we should not allow this.

          Show
          anoop.hbase Anoop Sam John added a comment - +1 for dev@ discussion.. Ya we should not allow this.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          We should figure it. I was trying to figure how to prevent random person creating system tables; has to be a Master CoProcessor at least. TBD.

          I test it just now. Unfortunately, it is ok to build table in System namespace. This issue is similar to bypass...Opening a thread in dev list is necessary as someone may have built their tables in System namespace.

          Show
          chia7712 Chia-Ping Tsai added a comment - We should figure it. I was trying to figure how to prevent random person creating system tables; has to be a Master CoProcessor at least. TBD. I test it just now. Unfortunately, it is ok to build table in System namespace. This issue is similar to bypass...Opening a thread in dev list is necessary as someone may have built their tables in System namespace.
          Hide
          stack stack added a comment -

          We should figure it. I was trying to figure how to prevent random person creating system tables; has to be a Master CoProcessor at least. TBD.

          Show
          stack stack added a comment - We should figure it. I was trying to figure how to prevent random person creating system tables; has to be a Master CoProcessor at least. TBD.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          We might not have to add this in hbase-default.xml I believe. If we have a section for compacting memstore in book, we can add abt this new config there.

          ok

          We allow this? I feel we should NOT allow users to add their tables to System.

          IIRC, use can do this. And we can't forbid it now as that will break the BC...

          Show
          chia7712 Chia-Ping Tsai added a comment - We might not have to add this in hbase-default.xml I believe. If we have a section for compacting memstore in book, we can add abt this new config there. ok We allow this? I feel we should NOT allow users to add their tables to System. IIRC, use can do this. And we can't forbid it now as that will break the BC...
          Hide
          anoop.hbase Anoop Sam John added a comment -

          We might not have to add this in hbase-default.xml I believe. If we have a section for compacting memstore in book, we can add abt this new config there.

          If users add their tables to System namespace

          We allow this? I feel we should NOT allow users to add their tables to System.

          Show
          anoop.hbase Anoop Sam John added a comment - We might not have to add this in hbase-default.xml I believe. If we have a section for compacting memstore in book, we can add abt this new config there. If users add their tables to System namespace We allow this? I feel we should NOT allow users to add their tables to System.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          +1. BTW, should we put this new config to hbase-default.xml? Or add it to our docs? If users add their tables to System namespace, they may complain the in-memory compaction setting from ColumnFamilyDescriptor is not working.

          Show
          chia7712 Chia-Ping Tsai added a comment - +1. BTW, should we put this new config to hbase-default.xml? Or add it to our docs? If users add their tables to System namespace, they may complain the in-memory compaction setting from ColumnFamilyDescriptor is not working.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Findbugs not from this patch. Ping for reviews.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Findbugs not from this patch. Ping for reviews.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 12s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 33s master passed
          +1 compile 1m 2s master passed
          +1 checkstyle 1m 2s master passed
          +1 mvneclipse 0m 26s master passed
          +1 shadedjars 7m 6s branch has no errors when building our shaded downstream artifacts.
          -1 findbugs 3m 14s hbase-server in master has 1 extant Findbugs warnings.
          +1 javadoc 0m 44s master passed
          +1 mvninstall 5m 47s the patch passed
          +1 compile 1m 8s the patch passed
          +1 javac 1m 8s the patch passed
          +1 checkstyle 1m 7s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedjars 5m 42s patch has no errors when building our shaded downstream artifacts.
          +1 hadoopcheck 58m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4.
          +1 findbugs 4m 22s the patch passed
          +1 javadoc 0m 50s the patch passed
          +1 unit 150m 49s hbase-server in the patch passed.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          259m 35s



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477
          JIRA Issue HBASE-18994
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893773/HBASE-18994_3.patch
          Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
          uname Linux 9ea21e39bc79 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 3cced09
          Default Java 1.8.0_141
          findbugs v3.1.0-RC3
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/9384/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9384/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9384/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 12s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 33s master passed +1 compile 1m 2s master passed +1 checkstyle 1m 2s master passed +1 mvneclipse 0m 26s master passed +1 shadedjars 7m 6s branch has no errors when building our shaded downstream artifacts. -1 findbugs 3m 14s hbase-server in master has 1 extant Findbugs warnings. +1 javadoc 0m 44s master passed +1 mvninstall 5m 47s the patch passed +1 compile 1m 8s the patch passed +1 javac 1m 8s the patch passed +1 checkstyle 1m 7s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 5m 42s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 58m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. +1 findbugs 4m 22s the patch passed +1 javadoc 0m 50s the patch passed +1 unit 150m 49s hbase-server in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 259m 35s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477 JIRA Issue HBASE-18994 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893773/HBASE-18994_3.patch Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 9ea21e39bc79 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 3cced09 Default Java 1.8.0_141 findbugs v3.1.0-RC3 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/9384/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9384/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9384/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Patch with suggested name for the config.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Patch with suggested name for the config.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          hbase.systemtables.memstore.compacting.type

          Am just noticing the difference - so you want it to be 'hbase.systemtables.compacting.memstore.type'? (memstore and compacting swapped) I see.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - hbase.systemtables.memstore.compacting.type Am just noticing the difference - so you want it to be 'hbase.systemtables.compacting.memstore.type'? (memstore and compacting swapped) I see.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 4m 15s master passed
          +1 compile 0m 38s master passed
          +1 checkstyle 0m 45s master passed
          +1 mvneclipse 0m 16s master passed
          +1 shadedjars 5m 25s branch has no errors when building our shaded downstream artifacts.
          +1 findbugs 2m 32s master passed
          +1 javadoc 0m 32s master passed
          +1 mvninstall 4m 25s the patch passed
          +1 compile 0m 48s the patch passed
          +1 javac 0m 48s the patch passed
          +1 checkstyle 0m 50s the patch passed
          +1 mvneclipse 0m 17s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedjars 4m 47s patch has no errors when building our shaded downstream artifacts.
          +1 hadoopcheck 44m 52s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4.
          +1 findbugs 2m 28s the patch passed
          +1 javadoc 0m 31s the patch passed
          -1 unit 122m 29s hbase-server in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          191m 10s



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477
          JIRA Issue HBASE-18994
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893705/HBASE-18994_2.patch
          Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
          uname Linux 020cf7ddf05b 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 93bac3d
          Default Java 1.8.0_141
          findbugs v3.1.0-RC3
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/9376/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9376/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9376/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 4m 15s master passed +1 compile 0m 38s master passed +1 checkstyle 0m 45s master passed +1 mvneclipse 0m 16s master passed +1 shadedjars 5m 25s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 32s master passed +1 javadoc 0m 32s master passed +1 mvninstall 4m 25s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed +1 checkstyle 0m 50s the patch passed +1 mvneclipse 0m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 47s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 44m 52s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. +1 findbugs 2m 28s the patch passed +1 javadoc 0m 31s the patch passed -1 unit 122m 29s hbase-server in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 191m 10s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477 JIRA Issue HBASE-18994 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893705/HBASE-18994_2.patch Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux 020cf7ddf05b 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 93bac3d Default Java 1.8.0_141 findbugs v3.1.0-RC3 unit https://builds.apache.org/job/PreCommit-HBASE-Build/9376/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9376/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9376/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 4m 8s master passed
          +1 compile 0m 35s master passed
          +1 checkstyle 0m 43s master passed
          +1 mvneclipse 0m 15s master passed
          +1 shadedjars 5m 8s branch has no errors when building our shaded downstream artifacts.
          +1 findbugs 2m 24s master passed
          +1 javadoc 0m 28s master passed
          +1 mvninstall 3m 49s the patch passed
          +1 compile 0m 37s the patch passed
          +1 javac 0m 37s the patch passed
          +1 checkstyle 0m 44s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedjars 4m 27s patch has no errors when building our shaded downstream artifacts.
          +1 hadoopcheck 36m 52s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4.
          +1 findbugs 2m 22s the patch passed
          +1 javadoc 0m 31s the patch passed
          +1 unit 103m 4s hbase-server in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          161m 56s



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477
          JIRA Issue HBASE-18994
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893692/HBASE-18994_1.patch
          Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
          uname Linux c4e74d77eeaa 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 93bac3d
          Default Java 1.8.0_141
          findbugs v3.1.0-RC3
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9370/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9370/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 4m 8s master passed +1 compile 0m 35s master passed +1 checkstyle 0m 43s master passed +1 mvneclipse 0m 15s master passed +1 shadedjars 5m 8s branch has no errors when building our shaded downstream artifacts. +1 findbugs 2m 24s master passed +1 javadoc 0m 28s master passed +1 mvninstall 3m 49s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed +1 checkstyle 0m 44s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 4m 27s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 36m 52s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. +1 findbugs 2m 22s the patch passed +1 javadoc 0m 31s the patch passed +1 unit 103m 4s hbase-server in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 161m 56s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:cb5c477 JIRA Issue HBASE-18994 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12893692/HBASE-18994_1.patch Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux c4e74d77eeaa 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 93bac3d Default Java 1.8.0_141 findbugs v3.1.0-RC3 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/9370/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/9370/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Again the naming consistency as pointed by Chia-Ping is not there?
          hbase.hregion.compacting.memstore.type vs hbase.systemtables.memstore.compacting.type
          So we can have hbase.systemtables.compacting.memstore.type ?

          Show
          anoop.hbase Anoop Sam John added a comment - Again the naming consistency as pointed by Chia-Ping is not there? hbase.hregion.compacting.memstore.type vs hbase.systemtables.memstore.compacting.type So we can have hbase.systemtables.compacting.memstore.type ?
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          I did not allow STring as the config because we need to convert the string to enum type so it may be error prone. But since generally this config won't be changed I allowed String config as suggested by Chia-Ping Tsai.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - I did not allow STring as the config because we need to convert the string to enum type so it may be error prone. But since generally this config won't be changed I allowed String config as suggested by Chia-Ping Tsai .
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          because we are discussing whether we can have memstore to be compacting but we don ask for the type of compacting memstore for these system tables.

          I assume users can set any compacting type for these system tables. It is ok to me.

          Show
          chia7712 Chia-Ping Tsai added a comment - because we are discussing whether we can have memstore to be compacting but we don ask for the type of compacting memstore for these system tables. I assume users can set any compacting type for these system tables. It is ok to me.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          I chose hbase.systemtables.memstore.compacting.type - because we are discussing whether we can have memstore to be compacting but we don ask for the type of compacting memstore for these system tables. So I prefer this.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - I chose hbase.systemtables.memstore.compacting.type - because we are discussing whether we can have memstore to be compacting but we don ask for the type of compacting memstore for these system tables. So I prefer this.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          It would be better to align it and the name hbase.hregion.compacting.memstore.type. Perhaps hbase.systemtable(s).compacting.memstore.type?

          Show
          chia7712 Chia-Ping Tsai added a comment - It would be better to align it and the name hbase.hregion.compacting.memstore.type . Perhaps hbase.systemtable(s).compacting.memstore.type ?
          Hide
          anoop.hbase Anoop Sam John added a comment -

          This config decides memstore impl class for all system tables right (not just META). Should we name it as systemtables.memstore? What we normally follow?

          Show
          anoop.hbase Anoop Sam John added a comment - This config decides memstore impl class for all system tables right (not just META). Should we name it as systemtables.memstore? What we normally follow?
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          hbase.systemtable.memstore.compacting.type

          Ya makes sense. This is better.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - hbase.systemtable.memstore.compacting.type Ya makes sense. This is better.
          Hide
          stack stack added a comment -

          I don't get that from the patch. Makes sense when you spell it out.

          Suggestion:

          hbase.systemtable.memstore.compacting.type

          If not set, just use default memstore type?

          Show
          stack stack added a comment - I don't get that from the patch. Makes sense when you spell it out. Suggestion: hbase.systemtable.memstore.compacting.type If not set, just use default memstore type?
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          If it is NONE - the default memstore is instantiated. If the config says do not use the default memstore type then we make the inMemoryCompaction type as BASIC which will instantiate the Compacting memstore and in Compacting Memstore BASIC is the default (since we also have EAGER).

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - If it is NONE - the default memstore is instantiated. If the config says do not use the default memstore type then we make the inMemoryCompaction type as BASIC which will instantiate the Compacting memstore and in Compacting Memstore BASIC is the default (since we also have EAGER).
          Hide
          stack stack added a comment -

          If default is set, we make compaction NONE, else we make it the 'default'?

          272 if (defaultMemstoreForSystemTables)

          { 273 inMemoryCompaction = MemoryCompactionPolicy.NONE; 274 }

          else

          { 275 // basic is the default type for compacting memstore 276 inMemoryCompaction = MemoryCompactionPolicy.BASIC; 277 }
          Show
          stack stack added a comment - If default is set, we make compaction NONE, else we make it the 'default'? 272 if (defaultMemstoreForSystemTables) { 273 inMemoryCompaction = MemoryCompactionPolicy.NONE; 274 } else { 275 // basic is the default type for compacting memstore 276 inMemoryCompaction = MemoryCompactionPolicy.BASIC; 277 }
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Simple patch. Any better name for the config?

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Simple patch. Any better name for the config?
          Hide
          stack stack added a comment -

          Pushing out. Pull in if gets patch.

          Show
          stack stack added a comment - Pushing out. Pull in if gets patch.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Sounds good to me.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Sounds good to me.
          Hide
          chia7712 Chia-Ping Tsai added a comment - - edited

          Can we just assign Default Memstore for all the system tables?

          That simplifies the issue but it doesn't bring any reason. Perhaps someone will raise another issue about "all system tables should be assigned to compacting memstore". As i see it, a workaround is to make it configurable, and the default memstore is the default setting.

          Show
          chia7712 Chia-Ping Tsai added a comment - - edited Can we just assign Default Memstore for all the system tables? That simplifies the issue but it doesn't bring any reason. Perhaps someone will raise another issue about "all system tables should be assigned to compacting memstore". As i see it, a workaround is to make it configurable, and the default memstore is the default setting.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Can we just assign Default Memstore for all the system tables?

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Can we just assign Default Memstore for all the system tables?

            People

            • Assignee:
              ram_krish ramkrishna.s.vasudevan
              Reporter:
              ram_krish ramkrishna.s.vasudevan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development