Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9870

HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.96.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      The patch makes sure that data block encoding in cache is always the same as that on disk. It deprecates the column family attribute ENCODE_ON_DISK since it doesn't apply any more. HColumnDescriptor#setEncodeOnDisk(boolean) is still there for compatibility issue. However, it doesn't do anything now. HColumnDescriptor#getDataBlockEncodingOnDisk() is also deprecated because the data block encoding on disk is always the same as that in cache for a given store file.
      Show
      The patch makes sure that data block encoding in cache is always the same as that on disk. It deprecates the column family attribute ENCODE_ON_DISK since it doesn't apply any more. HColumnDescriptor#setEncodeOnDisk(boolean) is still there for compatibility issue. However, it doesn't do anything now. HColumnDescriptor#getDataBlockEncodingOnDisk() is also deprecated because the data block encoding on disk is always the same as that in cache for a given store file.

      Description

      In this method, we have

          if (block.getBlockType() == BlockType.ENCODED_DATA) {
            if (block.getDataBlockEncodingId() == onDisk.getId()) {
              // The block is already in the desired in-cache encoding.
              return block;
            }
      

      This assumes onDisk encoding is the same as that of inCache. This is not true when we change the encoding of a CF. This could be one of the reasons I got data loss with online encoding change?

      If I make sure onDisk == inCache all the time, my ITBLL with online encoding change worked once for me.

        Attachments

        1. trunk-9870.patch
          137 kB
          Jimmy Xiang
        2. trunk-9870_v2.patch
          129 kB
          Jimmy Xiang
        3. trunk-9870_v3.patch
          139 kB
          Jimmy Xiang

          Issue Links

            Activity

              People

              • Assignee:
                jxiang Jimmy Xiang
                Reporter:
                jxiang Jimmy Xiang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: