Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10054

Add the default column compression option

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Abandoned
    • None
    • None
    • None
    • None

    Description

      Add the default column compression option for cluster level.
      If users don't set column compression for a column family, we should use the default column compression.

      Attachments

        1. HBASE-10054-trunk-v1.diff
          6 kB
          Shaohui Liu
        2. HBASE-10054-0.94-v1.diff
          3 kB
          Shaohui Liu

        Activity

          liushaohui Shaohui Liu added a comment -

          Patch for 0.94

          liushaohui Shaohui Liu added a comment - Patch for 0.94

          I don't see any real issue with that. But you should also update the documentation too.

          Can you do a trunk patch to trigger Hadoop QA?

          jmspaggi Jean-Marc Spaggiari added a comment - I don't see any real issue with that. But you should also update the documentation too. Can you do a trunk patch to trigger Hadoop QA?

          This code won't do anything as there is no real default value for the compression.

          ram_krish ramkrishna.s.vasudevan added a comment - This code won't do anything as there is no real default value for the compression.

          I guess that's the goal.

          If you don't configure any default value for the compression, then you keep the existing behavior.

          If you want to have a default value, then you add it in your configuration file.

          jmspaggi Jean-Marc Spaggiari added a comment - I guess that's the goal. If you don't configure any default value for the compression, then you keep the existing behavior. If you want to have a default value, then you add it in your configuration file.
          zjushch Chunhui Shen added a comment -

          IMPO, other similar configs could be added like this if we do so. It seems not a good start
          e.g. 'DataBlockEncoding', 'CACHE_DATA_ON_WRITE' and other common options in HTableDescriptor or HColumnDescriptor.

          We could do such a action on application code/script.

          Thus, +0 to add this option for me.

          About the patch,
          Nit comments:

          +          LOG.info("User donot set the compression for column fammily:"
          

          donot => don't

          Using desc#setCompressionType instead of desc#setValue because the former will skip the invalid compression type.

          Before set the compression to hcolumndescriptor, check whether the compression is available on the server, just like we do in HMaster#checkCompression

          zjushch Chunhui Shen added a comment - IMPO, other similar configs could be added like this if we do so. It seems not a good start e.g. 'DataBlockEncoding', 'CACHE_DATA_ON_WRITE' and other common options in HTableDescriptor or HColumnDescriptor. We could do such a action on application code/script. Thus, +0 to add this option for me. About the patch, Nit comments: + LOG.info( "User donot set the compression for column fammily:" donot => don't Using desc#setCompressionType instead of desc#setValue because the former will skip the invalid compression type. Before set the compression to hcolumndescriptor, check whether the compression is available on the server, just like we do in HMaster#checkCompression

          IMPO, other similar configs could be added like this if we do so

          I agree. I was thing the same thing. I don't think it's bad to have default values. Having none is already like having a default value that we can not configure... IMHO

          jmspaggi Jean-Marc Spaggiari added a comment - IMPO, other similar configs could be added like this if we do so I agree. I was thing the same thing. I don't think it's bad to have default values. Having none is already like having a default value that we can not configure... IMHO
          liushaohui Shaohui Liu added a comment -

          Patch for trunk.
          (a) add codec test at rs startup for hbase.column.compression.codec
          (b) update for the doc

          liushaohui Shaohui Liu added a comment - Patch for trunk. (a) add codec test at rs startup for hbase.column.compression.codec (b) update for the doc
          liushaohui Shaohui Liu added a comment -

          Update for zjushch's review

          liushaohui Shaohui Liu added a comment - Update for zjushch 's review
          liushaohui Shaohui Liu added a comment -

          zjushch

          IMPO, other similar configs could be added like this if we do so

          Compress is an important config, so it should have a default value to reduce the disk used for tables.

          liushaohui Shaohui Liu added a comment - zjushch IMPO, other similar configs could be added like this if we do so Compress is an important config, so it should have a default value to reduce the disk used for tables.
          zjushch Chunhui Shen added a comment -

          In the above comment, I mean test the compression in CreateTable if add the default comression.

          In modifyTable addColumnFamily events, should also add the default compression ?

          From the patch, I see the config "hbase.regionserver.codecs", what's the difference with the new config?

          zjushch Chunhui Shen added a comment - In the above comment, I mean test the compression in CreateTable if add the default comression. In modifyTable addColumnFamily events, should also add the default compression ? From the patch, I see the config "hbase.regionserver.codecs", what's the difference with the new config?
          liushaohui Shaohui Liu added a comment -

          zjushch
          "hbase.regionserver.codecs" is all codecs you want to check at the startup of regionservers
          If any check fails, regionserver will shutdown itself.

          liushaohui Shaohui Liu added a comment - zjushch "hbase.regionserver.codecs" is all codecs you want to check at the startup of regionservers If any check fails, regionserver will shutdown itself.

          People

            Unassigned Unassigned
            liushaohui Shaohui Liu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: