Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14511

StoreFile.Writer Meta Plugin

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had to modify the existing code of a StoreFile.Writer to add additional meta-info required by these new policies. I think that it should be done by means of a new Plugin framework, because this seems to be a general capability/feature.

      As a future enhancement this can become a part of a more general StoreFileWriter/Reader plugin architecture. But I need only Meta section of a store file.

      This could be used, for example, to collect rowkeys distribution information during hfile creation. This info can be used later to find the optimal region split key or to create optimal set of sub-regions for M/R jobs or other jobs which can operate on a sub-region level.

        Attachments

        1. HBASE-14511.v1.patch
          13 kB
          Vladimir Rodionov
        2. HBASE-14511.v2.patch
          13 kB
          Vladimir Rodionov
        3. HBASE-14511-v3.patch
          18 kB
          Jingcheng Du
        4. HBASE-14511-v4.patch
          20 kB
          Vladimir Rodionov

        Issue Links

          Activity

            People

            • Assignee:
              vrodionov Vladimir Rodionov
              Reporter:
              vrodionov Vladimir Rodionov

              Dates

              • Created:
                Updated:

                Issue deployment